3.5 Continuous improvement of programs and program components, and investigates opportunities for innovation, through an effective quality assurance system

The Quality Assurance Team (QAT) reviews data collected from many means to assess provider performance.  These include student testing data and assignment data from the Tk20 system. Depending on the issue being reviewed, QAT also includes non-traditional members who are off-campus partners. While this committee has existed in some form since 2013, as of 2019 the QAT has implemented a rolling calendar of data review.  At different points in the academic year, different program data will be reviewed by QAT and then shared with the entire unit during six annual retreats. This process will begin as of fall 2019. That schedule is outlined below:

The most prevalent shift in assessment strategy in the 2018 academic year was made to coincide with DESE updating the Annual Performance Report (APR) from version 1.5 to version 2.0.  The Department of Elementary and Secondary Education (DESE) developed a pilot system in 2016 to rate preparation programs in Missouri using completer performance data from a variety of assessments.  This was known as the APR 1.5, which had no accountability attached to it. The newer version, APR 2.0, was launched in 2018 and did include accountability measures. The data points that were collected and used to evaluate the program through APR 1.5 included:

• The Missouri Content Assessment, a multiple-choice content-area test that acts as a certification requirement
• Content Course GPA
• The Missouri Pre-Service Teacher Assessment, a performance-based assessment completed during student teaching
• The Missouri Educator Evaluation System, an observation form completed by university supervisors and cooperating teachers during student teaching
• First-Year Teacher Survey (Completer Results), a survey for program completers on how well their preparation programs prepared them for teaching.  It was completed a year after program completion.
• First-Year Teacher Survey (Supervisor Results), a survey for principals of program completers on how well preparation programs prepared their new teachers.  It was completed a year after program completion.

Only results from certification candidates, meaning program completers who successfully passed all assessments required for educator certification, are included in the APR.  Results from these data are then organized by the nine Missouri educator standards.

The APR 2.0 underwent modifications and was used to assess candidates completing in fall 2018 and spring 2019.  So, the QAT had to meet and discuss these changes and determine how assessment strategies needed to change across the unit.  

Updates to the APR 2.0 from DESE that were discussed at QAT meetings were:

1. The Missouri Pre-Service Teacher Assessment (MoPTA) was removed as the required performance assessment for Missouri teacher certification and as part of how educator preparation programs were evaluated in the APR 2.0.
2. The Missouri Educator Evaluation System (MEES) was revised and set as the new required performance assessment for Missouri teacher certification.  In addition, its impact on preparation program scores on the APR has been drastically increased. Points that were given for MoPTA performance have not been ascribed to the MEES.

Then, according to a points system based on these results, programs are awarded up to 100% of the possible points they could receive. 

 Results from these assessment placed programs into three categories.  These categories were:

• Accredited (70% or more of available points)
• Provisionally accredited (60-69% of the available points)
• Unaccredited (59% or less of the available points)

If a program receives less than 60% of the available points over the course of five years, DESE will make a recommendation to the State Board of Education as to whether that program’s ability to recommend candidates for certification should be revoked.  The State Board of Education maintains the authority to make the final decision in these cases.

All of Northwest’s programs were fully state accredited in 2018 and we have received preliminary DESE program approval for all teacher, leader, and counselor programs for 2019.  This data was reviewed at Professional Education Unit Retreats, Council on Teacher Education meetings, and Quality Assurance Team meetings. 

Beyond policy changes related to the APR, DESE enacted other assessment policy changes the QAT had to discuss.  These included:  

1. The potential switch to using ACT instead of MoGEA as a general education admission requirement was discussed as well
2. Updates to Missouri Content Assessments and policies were discussed, such as 
   1. Elementary education MoCA no longer being required for special education certification
   2. Updates to the elementary education MoCA, down to two subtests, as of Fall 2020
   3. Social studies MoCA will be reduced to one assessment as opposed to six subtests

These changes were discussed with the QAT and shared with program faculty as needed.

In November 2016, Northwest educator preparation programs received the honor of being invited to host nationally recognized assessment leaders on campus for the AACTE Data Systems Case Study. The purpose of the two day site visit was to chronicle Northwest's innovative use of data for continuous quality improvement of its educator preparation programs. The American Association of Colleges for Teacher Education sought to chronicle powerful practices to use data, information and analysis of quality assurance systems supporting clinical and field experiences, freshman advisement and assistance, the teacher education admissions and retention processes. Additional program goals were to identify mutually co-beneficial clinical partnerships, and the use of elementary student data in the laboratory school to prepare future elementary teachers. The visiting team, comprised of leaders, assessment directors, AACTE staff and leadership and state preparation program reviewers, offered praise and suggestions for continuous improvement. This site visit and case study began a larger conversation that led to changes in national accreditation systems, processes and expectations.

The Northwest educator preparation quality assurance system has consistently been acknowledged as a strength of the program, earning Target Standard for the previous national accreditation visits in 2005 (NCATE) and 2014 (NCATE), indicating a consistent application of human and financial resources to develop and implement high quality systems to track, monitor and support candidate learning and growth. Faculty and staff in the Northwest School of Education are currently editing a manuscript, Effectively Using Data for Educator Preparation Program Improvement, an AERA volume, as part of the Contemporary Issues in Accreditation, Assessment, and Program Evaluation Research in Educator Preparation. The genesis for this manuscript, which serves as the guidebook for a new assessment director in educator preparation, comes from the AACTE data systems case study work at Northwest in 2016. Since 2015, Faculty have presented more than 20 times on quality assurance and teacher education assessment at peer reviewed state, national and international conferences including AACTE, CAEP, AAQEP, AERA, MACTE, TECSCU, the Renaissance Group, and UCET.

Northwest has engaged in various strategies to ensure continuous improvement of program components.  These include:·

1. Since the MoPTA was dropped as a certification requirement and portion of the APR, all assignments and preparation materials from unit courses have been deleted.
2. In contrast, since the MEES has increased in importance for both candidates and programs, the unit focused on ensuring candidates were scored as accurately as possible.  Artifacts attached to the MEES have been developed by Northwest faculty and staff. These were developed and modified through unit retreats and QAT meetings. These will be used by university supervisors and cooperating teachers during student teaching to help them score initial candidates with the MEES summative form.   A pilot of MEES artifacts was implemented in the fall 18 and spring 19 semesters, scored by university supervisors. Also, content validity was calculated for these artifacts, which led to further revisions for these artifacts to be fully implemented for all initial candidates in fall 2019.  
3. Student-teacher surveys were revised by a subcommittee of the QAT, who updated surveys for initial candidates, cooperating teachers and university supervisors.  The initial candidate survey, in particular, was condensed from six surveys to one. Supervisors had their surveys reduced from two to one, and cooperating teachers from three to one.  This ensures that only actionable data are collected and reduces survey fatigue.  
4. In addition to student-teacher surveys, the use of student teaching data has improved immensely since fall 2018.  In fall 2018, there were multiple delays in the ability of cooperating teachers to access our LMS and score teacher candidates.  By meeting and partnering with the Office of Institutional Research, the unit devised strategies to improve the efficiency of data flow.  The result was that, in spring 2019, cooperating teachers were provided login data in record time and users were able to upload artifacts and assess each other with minimal issues.  This improvement process will continue through the summer of 2019 to increase the efficiency of student teaching data beginning when initial candidates complete their student teaching applications.  
5. Dispositional assessments for initial and advanced candidates have been reviewed and updated by committees outside of QAT. These results and their processes have been run through QAT (as described previously in Standard one).
6. A field experience tracking system was also unveiled and discussed at QAT meetings.  Programs needed to ensure that all candidates had an opportunity for clinical field experiences in diverse settings.  So, a subcommittee of the QAT developed a system to track field experiences to ensure that all students had experiences at diverse locations based on race, ethnicity, IEP, ELL, and SES.  This system was piloted in fall 2018 and spring 2019. The full implementation is planned for fall 2019.
7. The development of a unit dashboard for accreditation data was discussed and passed onto IT staff.  This dashboard will make assessment data, such as MoGEA, MoCA, Content GPA, First Year Teacher Survey and other assessments available by program to any education faculty on campus.  This will allow all on-campus stakeholders to view and use accurate data related to their programs and accreditation. The dashboard is in development and will be unveiled in fall 2020.
8. Qualitative data collection from current teacher candidates was discussed as well, and a survey was implemented in spring 2019.  Faculty felt that, while current student teaching surveys gathered a lot of data, that data was most typically too fine to look for program components that worked exceedingly well or were the most impactful to candidates.  As a result, faculty and QAT members partnered to develop an online survey focused on qualitative feedback. During an on-campus seminar for candidates who were student teaching, faculty explained the type of feedback they were seeking.  After a discussion, candidates completed the online qualitative feedback survey. The themes developed from that survey will be used to highlight Northwest’s most impactful preparation practices and will also help determine new directions the unit could take. 

Over the last several years, Northwest School of Education faculty and leaders engaged in a complete redesign of the educator preparation programs. Education redesign came out of the answer to a simple question: "If you could start from scratch and build the best teacher education program, what would it look like?" Given support and resources by the Provost and President, and with the help of PK-12 school administration partners, Northwest faculty completely overhauled the curriculum and clinical field experiences and assessment practices to attempt to strengthen an effective program while infusing earlier, more diverse, more robust clinical practice emphasizing the strength of diversity. Working collaboratively with faculty from Arts and Sciences, the professional schools, and the School of Education, and using data, evidence from PK-12 school leaders and recent program graduates, we have significantly altered the curriculum in several ways designed to positively impact P-12 learners' abilities to write, compute, reason, and share learning.

First, Education Redesign has increased all candidates' access to high-quality, diverse clinical practice (now more than 600 total hours) across all settings: urban, rural, and suburban, and with respect to all learner ability levels and representative of the vast diversity of the population of Missouri and the changing nation. Second, students engage in clinical practice by their third week of the freshman year. This significant shift changes the in-class conversation from a learner's point of view to that of a candidate-as-fellow-practitioner. To fully contextualize the meaning of clinical experiences and early observation, we had added a Professional Learning Community (PLC) in each semester. This is the third component of Education Redesign which has shifted the way candidates process their learning. All candidates, regardless of level or content specialty, teach and learn for three semesters in an integrated curriculum with embedded PLC guided by experienced professor-teachers to contextualize and help candidates reflect and grow from their experiences, successes, and occasional failures. Fourth, the redesign focused on improving candidates' formative and summative data use, instructional practice to engage all learners, differentiation of lesson plans to reach all P-12 learners, and classroom management strategies while heightening the primacy of content knowledge.

Northwest faculty have used data, including student feedback on courses, Professional Advisory Board Members' feedback, and pre-and-post survey data from candidates' experiences and diverse clinical placements, to better understand the impact of our program on candidate and P-12 student learning. This has resulted in several peer-reviewed publications and presentations. One example is the research article, "Teacher candidate perceptions of urban field experiences", which use data from our quality assurance system.

Another example of how the quality assurance system supports innovation is revealed in the presentation, "Relevant clinical experiences at a rural university in partnership with an urban school district: Perception of education teacher candidates serving in racially diverse, high poverty schools", which was presented at AACTE in 2018.

The impact of the quality assurance system on our ability to innovate comes from a faculty presentation at the Missouri Association of Colleges for Teacher Education (MACTE). This presentation, "Leading through change: Considerations and inspirations behind transformational redesign of educator preparation programs at Northwest Missouri State University", chronicled how we use data to transform and redesign our programs so that they would serve candidates better especially how they imported diversity, self-reflection and effective teaching strategies.

After Northwest began this redesign process the American Association of State Colleges and Universities (AASCU) awarded Northwest Missouri State University with the association’s  2018 Christa McAuliffe Excellence in Teacher Education Award.  Northwest, which also earned the McAuliffe Award in 2006, became just the third institution in the nation to earn the award two times since its inception in 2002.

In the application for the McAuliffe Award, the institution demonstrated an award-winning, innovative approach to curriculum redesign. Northwest added rich clinical experiences in with highly diverse school districts. Using data from our partners, alumni, and current students through the Professional Advisory Board we increased the number and duration of clinical experiences considerably. For more insight on how the Northwest educator preparation programs have used data from our quality assurance system, please see the award application.

The McAuliffe Award recognizes the culmination of Northwest’s work to overhaul the curriculum and clinical field experiences offered by its School of Education. The faculty-led redesign of the School’s curriculum placed greater emphasis on students’ access to diverse clinical practice – now totaling more than 600 hours – in urban, rural and suburban settings.  The redesigned programs involved extended partnerships with more than two dozen school systems, with an emphasis on assessment and instruction practices yielding high-impact results for student learning. The new curriculum also has Northwest students engaged in practice by the third week of their freshman year. Additionally, students participate in a professional learning community to learn from their experiences, while the curriculum focuses on improving students’ instructional techniques and classroom management strategies as well as multi-subject, integrated lesson plans.