Summer Matters program evaluation
How do summer learning programs impact youth development, academic success, and social-emotional growth?
Organization
Packard Foundation
Target audience
Packard Foundation
Target audience
- Packard Foundation leadership
- Campaign stakeholders
- Teachers, staff, parents, and youth at Summer Matters program sites
Overview
The Summer Matters Campaign aims to engage youth in safe, enriching summer programs that mitigate summer learning loss, enhance social skill development, and promote overall well-being. My work focused on evaluating these outcomes through a rigorous, mixed-methods research approach.
The Summer Matters Campaign aims to engage youth in safe, enriching summer programs that mitigate summer learning loss, enhance social skill development, and promote overall well-being. My work focused on evaluating these outcomes through a rigorous, mixed-methods research approach.
Role
As the lead educational program evaluator of the Packard Foundation’s Summer Matters Campaign, I analyzed data from three California summer programs to measure program effectiveness and provide actionable insights for improvement.
As the lead educational program evaluator of the Packard Foundation’s Summer Matters Campaign, I analyzed data from three California summer programs to measure program effectiveness and provide actionable insights for improvement.
Approach
My approach applied evaluation principles aligned with instructional design to ensure that analyses not only measured program impact, but also informed future curriculum development and instructional strategies. To that end, I designed and implemented an evaluation framework that examined four key areas:
My approach applied evaluation principles aligned with instructional design to ensure that analyses not only measured program impact, but also informed future curriculum development and instructional strategies. To that end, I designed and implemented an evaluation framework that examined four key areas:
- Program quality
- Social skill development
- Academic self-efficacy and motivation
- Impact on English Language Arts proficiency
Evaluation design
The evaluation design incorporated both quantitative and qualitative data sources to ensure a comprehensive analysis of program impact.
Working closely with partners at the Packard Foundation, I was able to get buy-in for this approach and ensure access to essential data sources, including school district data, participation records, and demographic data. I sent data requests to each district, and coordinated with district personnel to ensure the seamless transfer of anonymized data.
The evaluation design incorporated both quantitative and qualitative data sources to ensure a comprehensive analysis of program impact.
Working closely with partners at the Packard Foundation, I was able to get buy-in for this approach and ensure access to essential data sources, including school district data, participation records, and demographic data. I sent data requests to each district, and coordinated with district personnel to ensure the seamless transfer of anonymized data.
Data collection
To collect firsthand insights, I conducted on-site visits, fielded attitudinal surveys, and led interviews with program stakeholders, including parents, educators, and administrators.
I gathered parent and youth survey responses, and ensured that each program successfully administered standardized literacy assessments (specifically, the San Diego Quick Assessment) so I was able to map student progress in English Language Arts proficiency.
To collect firsthand insights, I conducted on-site visits, fielded attitudinal surveys, and led interviews with program stakeholders, including parents, educators, and administrators.
I gathered parent and youth survey responses, and ensured that each program successfully administered standardized literacy assessments (specifically, the San Diego Quick Assessment) so I was able to map student progress in English Language Arts proficiency.
Study scope
The study assessed both measurable academic outcomes (e.g., youth service hours, attendance rates) and social-emotional growth indicators (e.g., sense of safety, confidence, motivation), providing meaningful, relevant findings to Packard and the three school sites.
I compiled these findings into a report and companion deck, sharing data-driven recommendations to inform ongoing program improvements.
The study assessed both measurable academic outcomes (e.g., youth service hours, attendance rates) and social-emotional growth indicators (e.g., sense of safety, confidence, motivation), providing meaningful, relevant findings to Packard and the three school sites.
I compiled these findings into a report and companion deck, sharing data-driven recommendations to inform ongoing program improvements.
Impact
Findings from the evaluation demonstrated that high-quality summer learning programs contributed to sustained academic skills, stronger peer relationships, and improved work habits. These findings directly informed instructional design optimizations.
Findings from the evaluation demonstrated that high-quality summer learning programs contributed to sustained academic skills, stronger peer relationships, and improved work habits. These findings directly informed instructional design optimizations.
I was able to deliver these results by:
- Ensuring buy-in from funders, program leaders, and district stakeholders by keeping partners informed and engaged with the evaluation's progress.
- Conducting site visits across three program locations, engaging with stakeholders to establish relationships and collect rich qualitative data.
- Analyzing academic, demographic, and attitudinal data from over 1,000 youth participants, ensuring a rigorous evaluation process.