Monday, December 31, 2001

The Vermont Millennium Arts Partnership Evaluation, 1998-2001

The Vermont Millennium Arts Partnership (VT-MAP) is an Internet-based arts education program, providing an online forum to facilitate student learning through an exchange among students, teachers, artists, musicians, and other professionals. Several initiatives are involved in this project. The Vermont Musical Instrument Digital Interface Project (MIDI) was established in 1995 and Art Responding Through Technology (ARTT) was established in 1997 based on the model of the MIDI Project. The MIDI and ARTT projects were part of the predecessor initiative on which VT-MAP is based, the WEB Project. More recently, the Vermont Young Playwrights Online (VYP Online), a part of the Vermont Stage Company, joined this project in 2000. An offshoot of the MIDI project is the Digital Audio Initiative, which also joined the project in 2000. VT-MAP is a program of the Vermont Arts Council, operating in conjunction with MIDI, ARTT, VYP Online, the Vermont Alliance for Arts Education, and the Vermont Department of Education. Funding is provided through a grant from the U.S. Department of Commerce's Technology Opportunities Program, with other funding coming from the Jesse B. Cox Charitable Trust, Josephine Bay and C. Michael Paul Foundation, the State of Vermont, IBM, and Verizon.

In the last fiscal year of grant funding, from July 1, 2000 to June 30, 2001, VT-MAP awarded competitively funded grants to twenty-four schools (39 classroom teachers) in twelve counties throughout Vermont to participate in either the Vermont MIDI Project or ARTT. An additional ten schools participated in VYP Online, receiving funding to purchase software and attend teacher professional development training, and two additional schools received equipment to participate in MIDI's Digital Audio Initiative. Including the past two fiscal years, a total of 54 teachers in 34 schools have received grants and assistance through VT-MAP.

Classroom involvement in VT-MAP exposes students to the art, musical, and written creations of their peers, the language of critique in the Arts, the critique of their work by professional artists, and the creative possibilities to learn and share through the integration of technology into the classroom and class instruction. Students, teachers, administrators, and online mentors subscribe to a common protocol for sharing work online using a password protected website, which is based on the experience of online sharing undertaken by the Vermont MIDI Project in 1995.

The three projects focus their work and student learning on several Vermont and National Standards of Education, specifically the Vital Results of the Vermont Framework of Standards and Learning Opportunities. In classroom learning and the online network also address standards that focus on Reflection and Critique and Skill Development of young composers.

The Center for Rural Studies (CRS) at the University of Vermont is the evaluator of VT-MAP for the three-year grant period from July 1, 1998 to June 30, 2001. This evaluation is primarily based upon qualitative data collected through online and paper surveys, with supplemental quantitative data from surveys and website activity. The model of evaluation documents the project‟s implementation or process and resulting outcome of the project. Project process refers to the experience that takes place in the classroom and outcomes refer to the outcomes of teacher and student participation in the online forum and related in-classroom activities, specifically student performance and achievement based on Vermont and National Standards.

For more information about this study or to request additional copies of this report, please contact Michele Cranwell Schmidt, Evaluation Coordinator, at mschmidt@uvm.edu, call (802) 656-0256 or visit www.uvm.edu/crs/.

Thursday, May 31, 2001

The Teens Teaching Internet Skills Project

The Teens Teaching Internet Skills (TTIS) Project is a national initiative involving youth and adults from the 4-H Technology Teams in six states, including: Connecticut, Florida, Iowa, Maryland, Virginia, and Washington. This is a pilot project for a future, larger, national effort to engage youth in training individuals to navigate and obtain the needed information from the Medicare web site. The Center for Rural Studies (CRS) at the University of Vermont is the evaluator of the TTIS project. This evaluation report focuses on two distinct areas, including "best practice" of the process of the project and “learning outcomes” of the workshop participants, specifically whether or not the learner acquired computer and Internet skills to navigate and benefit from information on the Medicare website.

Based on qualitative data collected during the teen focus groups held at the National Training in Maryland in August 1999, the evaluators identified several initial expectations of the teens as well as ways to improve the training in the future. From the onset of their involvement, the teens felt there was a need for this program, as they felt that this project would serve to introduce or improve upon the computer and Internet skills of Seniors, allowing them to use and benefit from the information on the Internet. Initially, many of the teens did not have a definitive game plan, however, it was helpful for the teens to confer with one another, as they were able to gain a idea of how to initiate the project in their home state through this discussion. Many of the teens mentioned that it would be helpful in the future, to inform the teens prior to the training of the purpose of their involvement in the project and the focus of teaching Seniors Internet skills to search the Medicare website. The evaluators note that during future trainings, the participating Tech Teams should be informed of the importance of the workshop intake and evaluation forms, the need to use them, and how to use them correctly so representative data may be collected on the project’s impact on the learner.

Following the training in August 1999, from September 1999 to April 2000, the Technology Teams held a total of fourteen TTIS workshops, training one hundred and nineteen participants, most of whom were Senior Citizens and a few Care Givers. Based on qualitative data from the teens and adults, via a focus group questionnaire and periodic team updates, as posted on the TTIS listserves, the evaluators have identified several successful techniques that the teams have implemented for workshop preparation. These include identifying appropriate teens and training them to hold a workshop for Senior Citizens; developing and modifying an agenda depending on skill level, special interests, and size of the group; methods to advertise workshops based on the characteristics of the community; and finding suitable locations for conducting workshops, such as local schools, Senior Centers, and libraries.

The evaluation also reports on successful techniques used by the teams for handling a variety of learning situations. The teams reported that having a one to one ratio of participants to learners, use of small groups at workshops, being flexible to modify the agenda, and advertising for an audience with basic computer skills, are successful techniques for dealing with differing technological skill levels among learners. The teams also identified the use of ice-breaker techniques, such as playing solitaire and other computer games, as a helpful method to increase the comfort and confidence level of the participants in using the technology. They also noted that it is important to review with Seniors "next steps" at the end of the workshop, so the learners may continue to practice and benefit from their skills.

This report also includes a section on “lessons learned” from the pilot project for improvement of future workshops. The teams stressed the necessity of collaboration with local Senior Centers and schools for recruiting, advertising, and use of facilities for workshops. Further, it is important for teams to know the abilities and interests of their learners either before or at the beginning of workshops, in order to modify the agenda to meet the needs of the participants. The teams also mentioned that time commitment from both the adults and teens is a crucial element to the success of this project. This evaluation also reviews common problems encountered in conducting workshops and remedies for various situations, such as low attendance at workshops, technical problems, and other constraints such as time, travel, and the size of the technology team.

Further, this evaluation includes an analysis of the intake and evaluation forms that the learners completed before and immediately after the workshop, respectively. Although the use of the intake and evaluations were low among the teams, the results suggest that the target audience are primarily Senior Citizens who are eligible for Medicare (average age was 67), however Care Givers have expressed interest and attended a few workshops. Half of the respondents to the intake questionnaire have computer and Internet access in their homes, however few respondents reported using facilities that are open to the public. Further, approximately half report using computers on a daily basis and a third use the Internet daily. On the other hand, about a quarter of the respondents do not have access to a computer or the Internet, and the majority reported seldom or never using computers and the Internet. The respondents reported having minimal skills in using the computer, the Internet, and the Medicare website, however most people showed a desire to improve their skills, specifically searching on the Internet and using E-mail. About two-thirds of respondents were not aware of the tool “searching the Medicare website”. The major goals of the respondents for attending the workshop were to increase their comfort and skill level in working with computers and using the Internet in general. Almost a third of the respondents indicated learning how to use and search the Medicare website and to better inform themselves about Medicare, was a goal for attending the workshop.

A positive response was received from the learners, as over 80% of respondents indicated their skills with using computers and the Internet have improved after the workshop, compared to before the workshop where almost three-quarters of respondents doubted their skills at using the computer and the Internet. After the workshop, sixty-five percent indicated they were comfortable searching the Medicare website and half indicated a desire to search the Medicare website for information in the future. The learners were also pleased with the teaching ability of the teens and adults. Over 80% of persons said they would recommend this workshop to their friends and family.

The six month follow up with seven of the Seniors who agreed to be contacted and were able to be reached, indicate that their perceived comfort and skill level with computers has remained high following a longer period of time after the workshop. However, the results were not as positive for the Internet. This suggests that the TTIS workshops had an immediate and positive effect on Seniors perceived comfort and skill level towards technology. However, after a longer period of time following the workshop, the comfort and skill level decreased for some of the seniors in using the Internet.

Results from the post test of the project administered to the teen trainers indicate that the teens perceive a positive change in all skill areas, specifically working with Senior Citizens, skills related to project management and teaching, public speaking, leadership, and knowledge and understanding of the Medicare website. Further, this intergenerational experience led to a positive change in attitude of the teens towards working with Senior Citizens and teaching them skills in technology. This change in attitude included an increased comfort level and belief that Seniors are willing, able, and open to learning new technology.

This intergenerational experience fostered a positive attitude among the Seniors, with regards to their comfort in working with teens and the teens’ respect for, ability to teach, and patience with Senior Citizens. Overall, results suggest that intergenerational learning experiences benefit both parties in learning and skill development and led to a positive change in attitude towards the other generation.

For more information about this study or to request additional copies of this report, please contact Michele Cranwell Schmidt, Evaluation Coordinator, at mschmidt@uvm.edu, call (802) 656-0256 or visit www.uvm.edu/crs/.