» Accreditation » National Accreditation » Library
View Northwest Response to Jan 2020 AAQEP OSR 
1. Much presented evidence is directly dependent on or generated by state assessments. This is, of course, understandable; however, standardized assessments historically contribute to inequity by reproducing existing relations of power and privilege. Therefore, the following questions emerged in our review of the report:
Northwest remains deeply concerned about how state standardized assessments historically create roadblocks to candidates of color, candidates from impoverished socioeconomic backgrounds, and other candidates who otherwise have the potential to develop into great teachers but may not necessarily have strong test-taking skills.
Unit leaders and faculty recently had a great opportunity to interrupt the potential reproductive effects of standardized testing. As reported on page 109 of the QAR, Northwest candidates previously had to pass all four subtests of the Missouri General Education Assessment (MoGEA) to be admitted into the teacher preparation program. After receiving a memo from the Department of Elementary and Secondary Education (DESE), this requirement was amended. EPPs are now allowed to choose either the MoGEA or the ACT as an assessment of general education for admission to teacher education. With this freedom, Northwest endeavored to develop an admissions system that minimized racial and ethnicity inequality while maintaining as much academic rigor as possible.
As data was analyzed to maximize equity and rigor in the admissions process, gaps and concerns were brought to the surface. This analysis was outlined in Appendix G of the QAR. Some EPPs in Missouri made the switch to the ACT completely and removed the MoGEA as an admissions requirement completely. DESE recommended that, if EPPs switched to using the ACT as an admissions requirement, use the cut score of 20 on the composite ACT.
Analysis of Northwest data proved that the ACT had more racial bias than the MoGEA. From 2015-2018, 80% of White candidates reached an ACT of 20 or higher, compared to only 61% of Non-White candidates. By comparison, 94% of Non-White candidates passed all MoGEA subtests on their best attempt, compared to 95% of White candidates. The MoGEA proved to be more equitable than the ACT.
By adding a third assessment, GPA, the analysis outlined in Appendix G of QAR spread a wider net and ensured maximum equity and rigor. The final, three-tiered admission system included MoGEA passage, or an ACT of 20 or higher, or a GPA of 3.0 or higher. All candidates were still required to take the MoGEA at least once. By modifying this process, 98% of all candidates would have been admitted (compared to 95% when taking the MoGEA only), all candidates of color would have been admitted, if considering all candidates from 2015-2018.
While quantitative analysis was important, input into this process was taken from faculty with years of experience in mediating relationships between candidates and assessments. As noted on page 124 of the QAR, Mrs. Jill Baker was heavily involved in that process. Mrs. Baker has acted as the advisor for approximately 300-400 Elementary Education students for years before they are admitted into teacher education. While juggling these responsibilities, and helping candidates to navigate educator preparation program admission requirements, Mrs. Baker mediated relationships between candidates and assessments on a large scale. With that experience, she provided useful input into the revision process.
Institutional and state assessment data were analyzed in relation to one another to determine that the MoGEA was an inequitable assessment of general education. Since 2013, Dr. Tim Wall and Dr. Mike McBride have presented four times nationally and several times at state educator preparation education conferences on issues related to standardized testing inequalities. Specifically, the MoGEA assessment, when developed, demonstrated inequitable educational outcomes by comparing institutional and state data related to equity. This began in 2013 when Dr. McBride conducted an analysis of the precursor to the MoGEA, the C-BASE. This analysis indicated that on two of the subtests of the C-BASE, Science and Social Studies, that every potential cut score considered still ended with a statistically significant difference in the percent of White vs. Non-White candidates who passed these assessments. From there, Dr. McBride assisted Dr. Jeff Edmonds in data collection for his dissertation (Edmonds, 2014). This dissertation concluded that on the first version of the MoGEA, utilized from 2013 to 2015, there were statistically significant differences between African American and Hispanic candidate scores and those received by White candidates on all subtests. The assessment was then overhauled by Pearson for a 2015 relaunch.
The final step to ensure that equity is maximized regarding standardized testing for candidates is mentioned on page 109 of the QAR. The Teacher Education Admissions Committee (TEAC) reviews appeals if candidates do not meet any of the above admission requirements. Finally, the Dean of the School of Education can review appeals as well. All of these steps are provided to ensure equity of admission processes beyond standardized testing.
2. Because dispositions are deeply held beliefs that cannot be easily assessed with demonstrated behavior, the development of critical dispositions can yield results that are counterintuitive. For example, candidates may initially perceive themselves as wholly unbiased, and then through coursework and clinical experiences rate themselves lower. How are such complexities considered?
As first addressed in section 1.6 of Northwest's QAR, Northwest previously implemented a dispositions system based on the Niagara Candidate Disposition Inventory Sample. The Niagara included elements in which candidates self-scored themselves on 21 different dispositional elements; however, as our discussions regarding assessing dispositions evolved over the past two years, Northwest has now implemented a revised dispositions assessment, and the disposition assessment now relies entirely on faculty observation and evaluation.
The tool is still new, and data from the tool remains limited, but we believe the tool will lead to enhanced conversations between faculty advisors and students that will address some of the complexities described in your question.
It is also worth noting that how we define dispositions (at least at the state level) is still an ongoing conversation here in Missouri. There is presently a team for the state that is revising our MOSPE teacher preparation standards, and one of the conversations we have had regards replacing "dispositions" in the standards with "professional behaviors," and then focusing on tools then that measure more easily-assessed demonstrable behaviors in candidates.
This is not to say, though, that we do not think the more complex questions about candidates' self-perceptions of their own biases are not vital to good teacher preparation, but rather that we categorize that training instead under the umbrella of culturally responsive practice, which are addressed explicitly in our "Ecology of Teaching" and "Multiculturalism in Education" coursework, described in detail in section 1.3 of Northwest's QAR. As indicated, candidates in Multiculturalism in Education typically take the pre- and post-self assessment of diversity proficiencies. These reveal an examination of biases candidates may have related to diversity. In an analysis of Spring 2018 and 2019 data, 76% of candidates on the pre survey reported that they often consider they were "aware of my own biases and take them into consideration before I make a decision or act." On the post-assessment, after a semester of diversity experiences this rose to 96% (n=86).
The discussion of dispositions, here at Northwest, and throughout Missouri and the nation, is an important one, and we look forward very much to continuing that discussion with you during the site visit.
3. What data are being used to assess candidate development in relation to simulation experiences? How is this aim "enables the development of empathy and reinforces the need to build strong relationships with P-12 learners, teachers, and leaders to provide and support learning" assessed? How does the activity relate to professional dispositions and to what extent are root causes explored?
As noted on page 33 of our QAR, the Poverty Simulation is an activity completed during the fourth week of the 62-111 Ecology of Teaching course. This engaging activity allows deeper analysis and exploration of poverty from a larger macrosystem lens. The Poverty Simulation is a simulated activity where our candidates go through a live action role play with unexpected twists and turns reflective of those experiences that define day-to-day existence for those below the poverty line. This simulation transforms a few hours' time into four weeks living as a community member struggling through poverty. Our candidates and faculty find this to be a defining and enriching, and quite honestly a heart-breaking activity that enables the development of empathy and reinforces the need to build strong relationships with P-12 learners, teachers, and leaders to provide and support learning.
Several data sources are used to assess the development of candidates following this activity. These include a Freshmen Survey and also a reflection completed by the candidates.
The Freshmen Survey asks students to rate which of their experiences over their first year were the most valuable. In the 2018 results, 55% reported the Poverty Simulation, while 54% reported the same in 2019. However, open-ended questions on this survey provide in-depth insight into the impact of this activity. The other data source used to assess development is the reflection paper assignment associated with the Poverty Simulation. Responses from all of these sources are used to assess the development candidates experience from the Poverty Simulation and determine if any changes need to be made to this experience.
The Poverty Simulation is an excellent opportunity for candidates to develop more empathy. Here are some of the candidate responses from the 2018 Freshmen Survey indicating this:
From the Reflection Assignment, here are some sample responses that were used to assess how candidates developed empathy during the Poverty Simulation.
The simulation also helps candidates understand the importance of developing relationships with everyone involved in the P-12 student's macrosystem, as well as the importance of providing and supporting learning. This can be seen from the following responses to Reflection assignments:
The more complex question is, how is this experience related to dispositions? As assessed by Northwest, dispositions are considered professional behaviors. While this experience surely is an opportunity to exhibit professional behaviors, such as showing up on time and dressing appropriately, perhaps it is a better example of an opportunity to exhibit dispositions as defined by AAQEP. AAQEP defines dispositions as deeply held beliefs related to effective teaching. The Poverty Simulation has certainly demonstrated impact on deeper levels with our candidates.
From the Reflection assignment:
An even deeper analysis could be conducted to look at the root causes of poverty. From the Reflection assignment, some examples of potential discussion points could be:
During the application process for the McAuliffe Award, several candidates were interviewed about the most impactful experiences of their Freshmen year. One especially indicated that the Poverty Simulation was "huge" for helping her recognize privilege and how the background of P-12 students impacts their classroom performance and behavior.
4. Are there qualitative data indicating development of psycho-socio-cultural understandings? (Standardized assessments can be limited in evaluating such understandings and can in fact reinforce existing power relations and cultural/political dynamics.)
Northwest educator candidates complete a number of course assessments that require them to demonstrate their development of psycho-socio-cultural understandings. Some of these are described in the quality assurance report in the standard one section, where these topics are introduced. For example, beginning in their first semester on campus, candidates are introduced to learning theory in 62-111: Ecology of Teaching and 62-112: Developmental Foundations; in those courses, they complete summative assessment projects that measure their development quantitatively.
These courses emphasize the need for teachers to engage in culturally responsive practice and draw upon theory and research to inform practice. As outlined in the course learning outcomes in the Developmental Foundations syllabus (this syllabus and all other course syllabi are available in the Northwest AAQEP Canvas site), "candidates will apply their knowledge of development, theory, and research to define the essential components of culturally responsive practice." As part of their coursework, candidates write a personal narrative, which tasks them to synthesize their learnings on culturally responsive practice. 62-117: Inclusive Classrooms and Positive Learning Environments, generally taken in candidates' third semester in the program, further requires teacher candidates to fulfill a service project documenting fifteen hours of interaction with individuals with disabilities.
This coursework fosters cross-cultural understanding and teaches candidates multiple ways to consider and make meaning of culture. As foundation courses, these allow the teacher candidate to explore their own beliefs and biases. Further exploration of candidate understanding is explored in greater depth in the mid-level courses where students explore not only their own beliefs but see how their words and actions within the classroom can stifle or augment their P-12 students' in-class experience. This culturally responsive process is taught in the university classroom and then explored through either video review and/or field experience observations with intent for critical, reflective consideration. In order to afford our teacher candidates with the opportunity to consider cultural phenomenon that they did not grow up with, we ensure that the teacher candidates have field experiences outside of their own familiar experience (i.e. if a student was raised in an urban setting, they will have a field experience in the rural setting; if a student was raised in a rural setting, they will have a field experience in an urban setting).
Candidates then reinforce those concepts through later coursework, and then apply them in student teaching, which is assessed with the Missouri Educator Evaluation System (MEES) assessment tool, through MEES Standard #2.
5. How do school partners assess candidate readiness? How does this assessment compare to state surveys?
Both Northwest and our school partners lean on the MEES evaluation system in assessing candidate readiness. As discussed in section 3.2 of Northwest's Quality Assurance Report, all teacher candidates, in both the old and new programs, complete a culminating sixteen-week student teaching experience in grade-appropriate and content-appropriate classroom with a cooperating teacher. This exceeds the minimum state requirement of twelve weeks in student teaching. Cooperating teachers, as mandated by the state, are required to be certified teachers in Missouri, have a master's degree, have at least three years of teaching experience and certification, and meet teacher professional performance targets. In addition to their cooperating teacher(s), all candidates are assigned a university supervisor. Both the cooperating teacher and the university supervisor each make at least five formative evaluations and one summative assessment of candidate's teaching over the course of the sixteen weeks, using the MEES Teacher Candidate Assessment Rubric, as described in the MEES Protocols and Forms Guide. The MEES summative scores are reported to DESE, which then constitute one measure in each program's annual APR, as reported in Table 1: Northwest Teacher & Leader Preparation Programs Summary. In order to assure consistency, university supervisors and content supervisors attend annual MEES validity and reliability training days on campus prior to the start of each fall semester.
6. How is MOCA data for passing candidates used to support their development? How successful is the support system for candidates who failed prior to student teaching?
In order to earn certification in Missouri, educator candidates must pass a MOCA assessment. In order to identify students who struggle with the MOCA assessment, Northwest requires students attempt the MOCA prior to student teaching. If a candidate is unsuccessful, the clinical experiences director is notified, as well as the TESS office, and that candidate's advisor reaches out to the candidate to provide additional support. These supports vary based on candidate need but can include the Student Success Center (SCR), which provides test-taking strategy workshops and skills tutoring.
These supports have also included in the past, one-on-one sessions with faculty, including the Dean of the School of Education, who met with students and provided test-taking strategies and advice. One candidate, for example, who has been invited to meet with the visit team during the site visit, struggled during student teaching in the Fall of 2019 with the MOCA. After consultation, she met with the Dean of the School of Education, and on her second attempt successfully passed the MOCA assessment; similar stories from candidates abound in our programs.
7. What kinds of data are used in the leadership meetings? How are modifications based on data discussed in meetings tracked and assessed? (p.32)
School of Education Leadership meetings occur several times per month, and include the Dean/School Director and assistant director roles in these areas: curricular leader, state accreditation (DESE) leader, Coordinator-of-coordinator role, and operations/day-to-day. These roles were described on p. 20 of the QAR and explored in document, "descriptions of staffing and personnel". Other leaders provide input as well, including the assessment director (Dr. McBride), Assistant Director of Teacher Education ½ time role (Dr. Haughey), director of clinical experiences (Dr. Rich), certification officer (Ms. Hullinger) and Teacher Education Student Services Director (Ms. Wilson).
Data dispersed at these meetings include various APR measures, such as MEES summative results, MoCA data, First Year Teacher Survey (from both candidate and principal). The leadership team routinely looks at dashboard data, including student success and enrollment data, including; aggregate GPA; new program enrollment growth trends; program outcomes; DFWI rates; department graduation rate; retention; and data able to be disaggregated. Further disaggregation by total majors shows departmental advising loads. We also look at scheduling information, room capacity and facility conditions. We also look at the need for faculty and/or adjuncts.
8. What kinds of data are used in the advisory meetings? Are prompts topics determined by the university or the principal/superintendent groups? How are modifications based on data discussed in meetings tracked and assessed? (p.32)
Northwest makes use of a variety of data in its advisory board meetings. Section 4.4 of the QAR "AAQEP 4.4. Professional advisory board meeting- Data analysis from partners nov. 2019" reveals some more recent prompts provided to the Advisory Board.
School partners at the 2017 advisory board meeting were somewhat astonished by the redesign at its unveiling at an event supported by University President Dr. John Jasinski in September, 2017 to publicly unveil our redesigned program. Said one administrator of a partner school, "that's the kind of program I wish I would have had in college." Another indicated that Northwest actively used the feedback given at the previous advisory councils: "It's clear that they care about partnerships, quality, and improving. And they did what we told them they should do!" Partner feedback was essential in crafting the Education Redesign, and Northwest administration offered strong support at every stage. Further rationale for evidence to implement Education Redesign comes from our Unit Assessment System, which is in a constant state of renewal as we use assessment results to drive improvements.
Agenda and Data analysis of first-year teacher meetings: At the semi-annual Advisory Council meetings (both in Maryville and 100 miles away in urban Kansas City), our school partners asked for more evidence-based practice. They also offered to take our candidates and offer them instruction and assessment expertise based on current best practice. Thus, our redesigned courses emphasized four distinct content areas in the first year of candidates' experience: 1) first semester: Ecology and developmental foundations (with clinical experiences in diverse schools) 2) second semester: Instruction and Assessment (again featuring clinical experiences in diverse schools, this time in content areas like art, music, mathematics, and physical education). 3) Experience in a poverty simulation to enhance understanding of systems of inequality and issues of equity, and 4) integrated curriculum taught by professors of science, mathematics, language, and music, art, and physical education in a co-teaching setting with faculty from the School of Education. We believe that this initiative is unique in the country, and a model worthy of emulation, as it models to our candidates that co-teaching, and blending content knowledge and pedagogical content knowledge and teaching strategy and techniques are worthy of the work required to make them come together, across systems of faculty load credit, salary, and faculty evaluation. Removing those barriers with years of work is a practice that makes a transformative impact on the culture of the University, and models for students what we expect to see in their classrooms to integrate content and pedagogy."
"Northwest has endeavored to evaluate the impact of Education Redesign by utilizing statewide metrics and balance that data with candidate perceptions via case study, focus groups, interviews, and document analysis to triangulate results and achieve trustworthiness. Additionally, feedback from school partners at Advisory council meetings, first-year teacher surveys completed by administrators, and faculty perceptions, were synthesized to gauge the impact of Education Redesign on P-12 learners. This included multiple measures of qualitative and quantitative data."
