Organization

Ross Institute

Target audience

• Ross Institute leadership
  
• Ross School teachers
  
• Ross School staff

Overview

To optimize the annual Ross School teacher training, the Ross Institute assembled a cross-functional team of learning designers, program managers, and educators to reenvision a hybrid professional development initiative.

Role

As the learning evaluation consultant on this project, I was responsible for measuring program effectiveness and using data-driven insights to refine and improve the training experience.

Approach

My work included evaluating learner satisfaction, identifying areas for improvement, and ensuring alignment between instructional outcomes and professional development goals.

Curriculum development

Before starting on the evaluation, I first worked with other learning designers to develop the curriculum, define learning outcomes, and translate existing in-classroom professional development experiences into a web-based, asynchronous format.

We then met with the Ross Institute's leadership team to define elearning content for the annual training. I used this opportunity to begin mapping assessments for the elearning experience, and designing an evaluation for the in-person and asynchronous trainings.

I was able to get quick, iterative feedback on professional development program outcomes, which accelerated the evaluation design and prepared me to create an attitudinal survey.

Evaluation design

Once we locked the curriculum, I mapped educational outcomes to evaluation measures. I then designed a pre-post attitudinal survey using a Likert agreement scale, allowing participants to reflect on key learning experiences before and after training.

To streamline data collection, I mapped the survey to six domains to help organize incoming data: Content; Course; Design and Delivery; Learning Process; Instruction; and Facilitation.

Testing

To ensure the evaluation would be effective, I piloted the survey with Ross master teachers, gathering feedback to improve clarity and usability.

I then built the final evaluation in Moodle and collaborated with the team in a multi-round QA and content review sprint, ensuring the training and assessments were polished before launch.

Data collection and reporting

To maximize participation and ensure high-quality data collection, I joined the in-person training sessions and provided guidance on survey completion.

I instructed teachers on how to complete the survey and explained the benefits of survey completion. Being able to join in-person was truly a privilege because I was able to see the impact of the training firsthand, in real time.

After the training, I collected, cleaned, and analyzed the data, created data visualizations, and gathered my findings in an evaluation report and companion deck. I presented findings to leadership, and compiled recommendations for continuous program improvement.

Impact

A total of 47 respondents completed the pre-post survey, providing data that shaped ongoing improvements to the Ross Institute’s professional development initiatives. The evaluation confirmed high facilitator effectiveness and content engagement while also identifying critical areas for refinement.

Evaluation findings led to targeted program enhancements, including adjustments to content accessibility and instructional design improvements.

   Key findings include:

• Learners were satisfied with the course facilitators. 92% of learners reported that the facilitators were knowledgeable about course topics, while 83% reported that facilitators actively engaged them in learning.
• Learners were satisfied with the videos. Nine in ten respondents reported that the course included high-quality video (91.5%) and audio (93.6%).
• Learners were somewhat satisfied with the course content. The majority of learners reported content was logical (72.3%) and well organized (76.7%).
  
• Learners reported problems with accessing course materials, with just over half of respondents (55%) reporting the materials were easy to access.