Research
Team
|
Metis Associates brings over 35 years of experience in program evaluation, research, information technology, and grant development to its work with a wide range of organizations committed to making a meaningful difference in the lives of children and families. Metis conducted the random assignment study of the Framing Student Success project, documenting project implementation and assessing impacts on treatment school students and staff. The Metis research team included Susanne Harnett, Managing Senior Associate; Marisol Cunnington, Research Associate; and Angelina Lopez, Research Assistant.
|
The study was conducted in six NYC public elementary schools, all of which had been designated as Schools in Need of Improvement (SINI), identified as Title I schools, and served sizable English Language Learner and special education student populations. Each school also had a full-time visual arts specialist on staff who was responsible for teaching visual arts lessons aligned to the NYC Blueprint for Teaching and Learning in the Arts. Prior to project implementation, the six schools were randomly assigned to either the treatment or control conditions. In the three treatment schools, the project was first implemented with a cohort of third-grade students in 2009-2010 and continued through the subsequent two school years, until the end of students’ fifth grade year (2011-2012). A total of 545 treatment and 456 control students participated in any of the three project years, with a total of 266 treatment and 227 control students participating in all three project years.
Six schools meeting this set of eligibility requirements were identified during the project planning year and invited to participate in the study. All six schools signed a memorandum of understanding agreeing to participate as a treatment or control school prior to the random assignment process.
Key Study Findings
- The findings of the impact study of the Framing Student Success project indicate that it improved treatment students’ visual arts skills, literacy and math achievement, and Studio Habits of Mind.
- A regression analysis of change over time from 3rd to 5th grade, controlling for student characteristics such as race, gender, free/reduced price lunch eligibility, and special education and English Language Learner status, found that treatment students significantly outperformed control students in both ELA (2.2 scale score points) and math (8.3 scale score points).
- English language learners who participated in the program in 3rd and 4th grade also scored better on the Reading and Writing sections of their 4th-grade NYSESLAT tests than English language learners in the control schools. Performance of treatment students actually increased by 38.4 scale score points in Reading and Writing from 3rd to 4th grade, while the control group increased by 25.8 points. This difference is statistically significant and educationally meaningful (ES=0 .54).
- Treatment group students made significant and meaningful gains in the Studio Habits of Mind Engaging and Persisting, Stretching and Exploring, Observing, and Envisioning. Also, during their fifth grade year, treatment students made significantly greater gains than control students in their Reflecting skills.
- Three-quarters of the total sample achieved proficiency in visual arts by the end of the project. Thirty percent of students who were not proficient in visual arts in third grade scored at proficient (“meeting expectations”) or advanced (“exceeding expectations”) in fifth grade.
- Both visual arts and classroom teachers reported that the Framing Student Success project gave them the opportunity to collaborate more frequently with their colleagues. Furthermore, they intend to continue their collaboration, though they do acknowledge heavy teaching loads and scheduling constraints pose a challenge.
- Teachers who collaborated on a weekly basis with a professional artist to teach arts lessons integrated with math and ELA reported greater job satisfaction by the end of the year, and reported greater skills in the area of discussing arts achievement with students. Both teachers and school arts specialists reported a greater likelihood of implementing arts-integrated lessons in the future.
- The classroom teachers became more comfortable teaching visual arts over time, and began to see the value integrating the visual arts into the core curriculum.. They saw how well the visual arts could engage students and help them understand academic subjects. Teachers reported that the project made them more aware of the value of art as an “access point” for other subject areas, and helped them see opportunities to incorporate visual arts within the school curriculum, particularly in longer-term units of study. They also valued the opportunity for students to be creative and develop skills in new areas.
- Principals reported that they were more aware of how to supervise the arts at their school because of their participation in the program. Furthermore, they reported that the project heightened their awareness of the capacity of the arts to engage parents in their children’s academic and artistic development.
