1.4 Assessment of and for student learning, assessment and data literacy, and the use of data to inform practice

Throughout the university, coursework in both the Professional Education Unit (PEU) and also in general education emphasizes gathering, using, and analyzing information through one of the university’s seven institutional learning outcomes: Managing Information. The Managing Information outcome states that all Northwest students will be able to “locate, retrieve, organize, store, synthesize, and annotate various forms of information,” and “evaluate information for currency, usefulness, and accuracy.” Through these core university-wide outcomes, Northwest graduates receive support, education, and practice to use information wisely and discriminate between valid and reliable information- a particularly useful skill in the current environment.

Faculty introduce concepts related to assessment and data literacy early in all education programs. Beginning with the basics of assessment in introductory locations, including the cyclical nature of assessment and its position in lesson planning, and the role of standards and grade-level expectations in planning for and delivering assessment. Students are also introduced to assessment delivery, data analysis, data teams, and how data informs future instruction. Students are exposed to principles of effective assessment in each program, including formative and summative assessment and informal and formal assessment. 

Assessment and data literacy are introduced in one of two locations: through a shared course for a majority of teacher certification programs or through stand-alone course locations in three programs for introducing assessment and data literacy. Specifically, these strategies are deployed in specific coursework in the teacher preparation program at the undergraduate level, including 62-115 Principles of Assessment, 19-180 Introduction to Music Education, 22-260 Observation and Practicum I in PE, and 03-320: Foundations of Agricultural Education. The following course description aligns well with AAQEP standard 1.4, which addresses data and assessment literacy and the use of data to inform practice. At Northwest, all education majors complete a Principles of Assessment course by the end of their first year, and practice honing their craft:

Syllabi for these courses (and all other PEU courses) are available in the Northwest AAQEP Canvas site.

Reinforcement of assessment and data literacy begins in Phase II of the educator preparation program. All students are required to enroll in 62-321 Designing Interventions and Assessment. In this course, students will scaffold their introductory assessment coursework to consider alternative assessment types to meet all student needs. This includes special populations of students such as ELL, special education, and/or who are academically talented or gifted. Students are concurrently enrolled in content methods courses. Some areas have additional assessment-specific content in their programs, including agriculture and physical education. Content methods courses are specifically designed to include practices in assessment, as noted in the course description or course syllabus.

Early Childhood undergraduate candidates take 62-303: Observation and Assessment in Infant Toddler Settings, 62-403: Observation and Documentation as Assessment in Preschool and Kindergarten, and 62-426: Integrated Curriculum and Authentic Assessment for Evidence-based Decision Making Grades 1-3. Early Childhood and Elementary candidates take 62-322: Literacy Assessment Intervention: Theories and Techniques. 

Agricultural education students reinforce assessment and data literacy in 03-420 Program Planning in Agriculture Education; 03-524: Methods of Instruction and Assessment in Agricultural Education Laboratories and 03-580: Methods of Instruction and Assessment in Agricultural Education.

Syllabi for these courses (and all other courses throughout the Professional Education Unit) are available in the Northwest AAQEP Canvas site.

Please note: Missouri Educator Evaluation System (MEES) is introduced in an earlier section of this report. If you have not already read that subsection, it is recommended that you go back to the previous section which details AAQEP Standard 1.1. This section will only address the MEES assessment as it applies to MEES Standard #7.

Because assessment and data literacy, professionalism, and professional collaboration are difficult for a university supervisor to assess through observation, Northwest faculty began collecting artifacts in 2018 to measure teacher candidate performance on MEES Standard #7 (assessment artifact), MEES Standard #8 (professionalism artifact), and MEES Standard #9 (professional collaboration artifact). These three artifacts then provide university supervisors insight into candidates’ skills in assessment, their professionalism, and their professional collaboration during their student teaching. (Please note that information about how Northwest uses Standards #8 and #9 data in its continuous improvement can be found earlier in this report in the section detailing AAQEP Standards 2.5 and 2.6) Northwest is closely monitoring the collection of these artifacts and seeking candidate feedback as it strives to continuously improve this artifact collection; this is a new process for Northwest and will likely be improved in the coming years, but it is an innovative approach to measuring MEES 7, 8, and 9, and Northwest takes great pride in its strides thus far in the artifact collection.

MEES Standard #7 explicitly addresses candidates’ capacity for assessment and data analysis, which has (as well as the other eight) been cross-walked throughout all of the university’s education programs, as outlined in Table 3: Northwest State and National Educator Prep Standards Alignment. As shown in the data below, Northwest candidates’ performance on MEES Standard #7 compares favorably with the state average for 2018-19.

Below are summative scores from initial candidates who student taught in the Fall 2018 and Spring 2019 semester:

Further information regarding Northwest candidates’ performance on summative MEES assessments is available in later sections in this report.

The Missouri Department of Secondary and Elementary Education (DESE) has a suite of six surveys educators complete in their first year of employment, which DESE has been collecting since the 2015-16 academic year. The First-Year Teacher Survey (FYTS), taken by teacher, in alignment with MEES Standard #7, asks completers five questions after their first year of teaching to assess their preparation in assessment and data literacy (FYTS questions 30-34). In addition, the First-Year Teacher Survey by principals (FYTS), taken by principal, which is completed by principals assessing Northwest teachers following their first year on the job, likewise has four similar questions regarding student learning (questions 29-33). In the first of these, as shown in the table below, 88% of completers in their first year and 81% of their principals “agreed” or “strongly agreed” that in their first year they were “prepared to use assessments to evaluate learning.” They also indicated that they felt prepared to “develop” and “analyze assessment” in improving student learning, as well “help[ing] students set learning goals” and “work[ing] with colleagues to set learning goals”. In all of the questions, Northwest-prepared candidates perceived themselves to be better prepared than first-year teachers at peer institutions. In four of the five questions, new completers’ principals perceived them to be stronger than first-year teachers at peer institutions. 

Trends in these data over the past four years, as shown in the two charts that follow, have remained relatively consistent overall over the four years in which Missouri has been administering the FYTS.

As a result of this data (and the accreditation self-study process), Northwest is taking steps, to be discussed in more detail in the “Conclusion: Findings and Recommendations” section of this report, in the 2019-20 academic year to provide a stronger finish to the old programs. In 2021-22, the first completers from the newly redesigned undergraduate programs and their principals will take the FYTS surveys, and Northwest looks forward to comparing their data to these current completers’ data. Below, please find graphical representations comparing and contrasting principal and completer views related to differentiating and modifying instruction for P-12 learners.

All candidates in our Master’s and Specialist programs take at least one course that addresses assessment and data literacy. Northwest programs at their heart believe that assessment and data literacy are foundational concepts in any advanced student of education. Syllabi for courses described below (and all other syllabi from PEU classes) are located on the Northwest AAQEP Canvas site. Additional information (beyond that described below) about Northwest candidates' clinical experiences related to assessment can be found in Table 4: Northwest Teacher Clinical Experiences Crosswalk.

Candidates in the Online Professional advanced programs (Educational Leadership M.S.Ed., Curriculum and Instruction M.S.Ed., Special Education M.S.Ed., Early Childhood M.S.Ed., and Reading M.S.Ed.) take 61-461: Data Literacy, in their Master’s Program. This course prepares students to follow best practices in data literacy. They define district or classroom problems; locate literature that provides evidence on successful interventions for the identified problem, select key information about the problem that could be studied and where action could be taken. Graduate students find, select and organize data about the key information; run qualitative or quantitative analyses; present on findings; and reflect on how this experience and new skills will improve their teaching, leadership, and directly (and positively) impact their schools, classrooms, and students.

The course begins with a definition of data literacy and the importance of tracking data formatively as opposed to summarily. Candidates work through case studies to identify various types of qualitative and quantitative data available in schools and work to narrow down which data is most effective to recognize student achievement and success. Candidates then conduct a minor literature review and also track data related to an initiative in their own building(s), such as a teaching strategy or intervention practice being emphasized in the school or district. Candidates conduct a literature review and identify three pillars of support from relevant scholarship, and then design an effect-size study with the student data in their school or classroom. As part of the course, students are introduced to descriptive statistics and various forms of data analysis, including correlation, T-Tests, ANOVA, and the Cohen D effect size they use in their own research design.

Students create a culminating project called a data wall to present the findings of their research student that outlines the background, methodology to their study, purpose, results, and analysis of the study they conducted. The data wall concludes with the student's idea of what aspects should be analyzed in future study around the topic they have been investigating. Over the seven-week period, candidates have conducted an in-house research study that directly impacts their students and prepares them for a leadership role in which they help other teachers in their buildings to create their own research within their classrooms.

At the graduate level, each Master’s program shares the 61-641 Data Literacy Course. We believe all candidates should possess knowledge and skills in data analysis. Analyzing real-world data, finding improvements, and reporting those findings to administrators shows the Northwest support for mutually-beneficial partnerships, improvement-oriented scholarship, and academic learning through advancing content knowledge. Assessment and data literacy, along with Multicultural Education for Diversity, Equity, and Inclusion are the core courses that define our brand in graduate education.

Candidates in the Specialist-level programs take 61-723: Writing and Interpreting Educational Statistics. The course builds on fundamentals of assessment and data literacy introduced in the candidates' previous degree(s) and coursework. Having a two-fold focus, course activities and assignments reinforce (1) methods of developing and reporting educational research and literature and (2) techniques of interpreting and evaluating educational research and literature. Basic statistical methods are utilized in each of the components. Specialist candidates apply their assessment and data literacy knowledge and skills in developing, implementing, and writing a research project. In this project, they choose a statistical test to use for a given data type and situation; locate datasets to test research questions; clean a dataset; run descriptive analysis; run parametric and non-parametric statistical tests; interpret statistical test results; and write measurable, repeatable, quantitatively-focused research questions. They also write a purpose statement that aligns with research question(s); write a literature review with three main pillars of references; write an effective methodology section; write a findings and results section; and elaborate on conclusions drawn from the literature review. Finally, the student will learn how to become a data literacy leader and recruit colleagues for data-based collaborations. This project replaced a former Thesis-style, five-chapter Field Study, which was widely viewed by students and district partners as too “Ivory Tower”; unrealistic, and detached from the daily work of school leadership. At Northwest, we selectively let go of the past as we strive for progress through innovation and partnership.

Candidates in the School Counseling Master’s Program take courses to design, implement and evaluate a comprehensive school counseling program that supports students' academic, career, and social/emotional development. Candidates in the program take the following courses that explicitly address assessment:

• 68-652: Group Appraisal introduces counselor candidates to assessment and addresses the broad area of measurement, including an examination of psychoeducational appraisal methods. Selection, administration, scoring, interpretation, and avoidance of bias in the use of group appraisals are also studied.
• 68-671: Student Appraisal reinforces assessment, covering the basic psychometric criteria for appropriate appraisal, reading and interpreting psychological reports, the principles of selection of cognitive and non-cognitive appraisal instruments, and the federal and state regulations for placement in special education.
• 68-660: Organization and Administration of the K-12 School Counseling further reinforces assessment; the course is a study of the processes fundamental to the establishment and administration of the comprehensive developmental school counseling program-emphasis on the use of needs assessment, time-task analysis, and student learning outcome data to plan, implement, and monitor school counseling programs.
• Candidates then apply the principles of assessment learned in the three above courses in 68-640: Group Counseling, 68-636: Research Methods, 68-628: Supervised Practice in School Counseling, 68-639: Research Project, and 68-629/630: Practicum in Elementary or Secondary School Counseling.