CORE + Prosody is a comprehensive measure of reading fluency that unites accuracy, rate, and prosody of students’ oral reading.
Oral reading fluency is an essential part of reading proficiency1, and is perhaps the most prevalent reading assessment used in classrooms across the U.S.; however, these traditional assessments measure only accuracy and rate, and entirely neglect prosody. Prosody is reading with appropriate expression and phrasing, and is one way a reader demonstrates they understand the meaning of the text. Research has shown prosody to be an important indicator of comprehension, beyond accuracy and rate alone, particularly among developing readers2.
The purpose of CORE + Prosody is to develop and validate an automated scoring system to measure, unite, and scale the accuracy, rate, and prosody in oral reading fluency to be used as a screening and progress monitoring measure for students in Grades 2 through 4.
CORE + Prosody is a four-year project funded by the Institute of Education Sciences (IES), the statistics, research, and evaluation arm of the U.S. Department of Education.
Apply a machine learning model to measure and score reading prosody.
Develop a psychometric model to produce a comprehensive measure of fluency that includes prosody in addition to accuracy and rate.
Apply this psychometric model to a scaling framework by calibrating passage parameters on a common scale across grade levels (i.e., vertically linked), providing an advantage for across-grades growth monitoring of students oral reading fluency.
Analyze the relation between accuracy, rate, prosody, their combined scale score, and reading comprehension and cognitive load to determine the incremental influence of prosody on comprehension skills and general reading proficiency.
Enhance the interpretability and usability of the project assessment for educators.
CORE + Prosody has the potential to increase the reliability and validity of decisions made from oral reading fluency assessment scores, resulting in better identification of students in need of reading interventions, and better evaluation of those interventions.
The CORE + Prosody assessment:
computer-administered and -scored to reduce lost instructional time and administration costs
offers a measure of prosody and a unified oral reading fluency score of accuracy, rate, and prosody to align with reading theory and provide educators with more meaningful reading information
provides a vertical scale across grades to improve score reliability and progress monitoring accuracy
Computerized Oral Reading Evaluation - CORE
CORE + Prosody builds upon Computerized Oral Reading Evaluation (CORE), a project that uses (a) shorter passages, (b) automatic speech recognition to score oral reading fluency accuracy and rate, and (c) a latent variable psychometric model to scale, equate, and link scores across Grades 2 through 4 to improve reading outcomes for students across reading proficiency levels. Please visit the CORE project blog for information about the project procedures and results.
CORE uses automatic speech recognition to score oral reading fluency accuracy and rate.3
Alleviates the resource demands of one-to-one testing administration and the resource cost to train staff to administer and score oral reading fluency assessments
Reduces the time cost of oral reading fluency administration by allowing small-group or whole-classroom testing
Reduces human ORF administration errors by standardizing administration setting, delivery, and scoring
CORE uses a latent variable psychometric model to scale, equate, and link scores across Grades 2 through 4.4
The model-based scores are on the same metric as traditional oral reading fluency scores: words correct per minute (WCPM). This makes the scale scores immediately usable for teachers and reading specialists who are familiar with the words correct per minute expectations for students at specific times in specific grades along the reading continuum.
The model-based words correct per minute scores offers educators scores that are comparable regardless of the passages read, and is a vast improvement on the common practice of using readability estimates (e.g., Flesch-Kincaid) to equate passages.
Can mitigate the constraint of grade-level assessments by applying a common model-based words correct per minute scale across grades, providing an advantage for growth analyses of student oral reading fluency across grades.
Reduces standard error of oral reading fluency measurement 5, particularly for low-performing students, improving the reliability of scores and yielding scores sensitive to instructional change.
The model-based words correct per minute scores can increase the reliability and validity of the decisions made from scores, yielding better identification of students in need of reading interventions, and better evaluation of the results of those interventions.
Developing Computational Tools for Model-Based Oral Reading Fluency Assessments
This project expands upon the existing estimation model developed as part of the CORE project to include the development of (a) a sentence-level model that takes into account between-sentence dependency, and (b) incomplete reading. The model parameter estimation algorithms for these extensions will be developed by the method of moments, the Monte Carlo EM algorithm approach, and the Bayesian HMC approach used in the software package Stan.
This project will produce a Shiny app for rendering user-friendly R code needed to estimate model parameters, providing better oral reading fluency score comparability both within- and between-students for better longitudinal and cross-sectional studies, as well as better estimates of measurement errors for oral reading fluency scores. The research team will demonstrate the use of the new software on data collected by the CORE project, and develop web-based tutorials for supporting applied researchers who want to use the Shiny app.