ASSESSMENT
Beyond Averages: Using NAEP to Better Understand Student Reading Scores
Go deeper into the data to discover more of the story.
By Marisol Kevelson, Katherine E. Castellano, Emily Kerzabi, and Eli Holder
Our team is committed to innovating student group score reporting to support improved learning opportunities for all students. We began our collaboration as part of a co-design team of policy-focused assessment data users and researchers who collaborated to develop better displays of widely used assessment results. We led the team as partners—ETS researchers and data visualization expert Eli Holder (3iap), who had recently published the groundbreaking article Unfair Comparisons. Last spring, we met biweekly to conceptualize different ways to report student group test results that would move the reaction from “That’s just how it is” to “We can help students by addressing needs X, Y, and Z!” Throughout that experience, a mantra that guided all our designs was “Go beyond the average.” In this blog, we teamed up again to share how that simple change can illuminate so much more about student performance.
For years now, students in Massachusetts have been outperforming students in other states, on average, on the National Assessment of Educational Progress (NAEP) reading assessment. They did so again in 2024 for both grades 4 and 8. Such high marks have caught the eye of policymakers and education leaders in other states who use Massachusetts’ results as a benchmark and for hints to guide their own state policies. Mississippi and Louisiana, two states that historically scored much lower than Massachusetts on NAEP reading, have been lauded for their NAEP reading score gains, becoming examples for other states looking to their policies for ideas. Yet, while Mississippi and Louisiana have climbed up the NAEP state rankings, their average scores are still lower than states at the top like Massachusetts. Digging more deeply into NAEP data and going beyond averages can give us a clearer and more complete understanding of what Mississippi and Louisiana have achieved and how their students' literacy levels compare to each other and to Massachusetts.
Figure 1. A bar chart of average Grade 4 reading scores on the 2024 National Assessment of Educational Progress (NAEP) shows that Massachusetts scored higher, on average, than Louisiana, Mississippi and the nation.
The NAEP Data Explorer (NDE), which provides users with menu-driven access to a treasure trove of NAEP data, allowed us to dig into state-level student reading performance for 4th grade students. Figure 1 provides a bar chart of the average Grade 4 NAEP reading scores for the nation and for Louisiana, Mississippi, and Massachusetts. While bar charts like these were once a popular convention, they’re misleading in many ways, in particular leading to deficit thinking. We note a few observations from this simple bar chart:
- Massachusetts students have the highest average Grade 4 reading performance in the United States, with an average score of 225, a full 10 points higher than the national average of 215 for 2024. This difference is not random noise. It is statistically significant, as are all other differences noted throughout the blog.
- Mississippi had the next highest average score of the three states with 219, which is 6 points lower than Massachusetts but 4 points higher than the nation.
- Louisiana’s average score of 216 was similar to the nation’s overall.
In addition to averages, the NAEP Data Explorer provides state-level data by several percentiles. Percentiles divide a sample of students into 100 equal parts, each representing 1% of the students. When we talk about a specific percentile, we are referring to the value at or below which a certain percentage of the students fall. For example, if a student’s score is at the 75th percentile, it means their score is equal to or higher than 75% of all other students’ scores. Plotting several percentile points helps us better understand how scores compare across the entire distribution of students.
Figure 2. Annotated distribution plot of 2024 Grade 4 NAEP reading scores for the nation.
From left to right, the panels in Figure 2 show how additional layering of information provides insights into student scores in relation to the NAEP scale.
- In the left panel, we see how a simple bar chart only conveys a single point—the national average. In charts like these, users tend to make up their own idea of the spread of scores within or around the bar, which can lead to misunderstandings of the data and of why some groups have lower scores than others.
- In the middle panel, we use a score distribution plot to highlight where students tended to score. Here, the darkest shaded section shows that the middle 50% of students nationwide scored from 189 (25th percentile) to 245 (75th percentile).
- The right panel adds in the NAEP achievement levels to provide a point of reference for interpreting the score percentiles. The achievement levels describe how well students demonstrate knowledge and skills, with NAEP Basic representing partial mastery. Here, the darkest shaded section shows that the middle 50% of student scores spans three performance levels: Below NAEP Basic, NAEP Basic, and NAEP Proficient. Additionally, the 90th percentile is just shy of the NAEP Advanced level, meaning that less than 10% of students scored high enough to reach the advanced level.
Figure 3. Distribution plots of 2024 Grade 4 NAEP reading scores. Average reading scores for the nation and each state are shown in white. Dashed vertical lines indicate the percentile points for the national distribution.
Take a look at Figure 3. Do you see what we see? The left side of the distribution for Mississippi resembles that of Massachusetts more so than that of the nation or Louisiana. Mississippi’s and Massachusetts’s 10th percentiles are about 10 points higher than the nation.
Both Mississippi and Massachusetts are seeing
better scores for lower performing students
than the nation overall.
What do you see at the right end of the distribution? There, Mississippi scores are lower than those of both Massachusetts (by 13 points) and Louisiana (by 4 points).
Mississippi’s highest performing students
may need more supports to reach the
same levels of performance as students in other states.
NAEP also collects data on several student characteristics, such as whether students’ families are economically disadvantaged. When we break out scores by economic status, we see a more complete picture of 4th grade reading scores in Mississippi, Louisiana, and Massachusetts (see Figure 4).
Figure 4. Distribution plots of Grade 4 NAEP reading scores broken out by family economic status. Average grade 4 reading scores by family economic status for the nation and each state are shown in white. Dashed vertical lines indicate the percentile points for the national distribution.
When analyzed by whether students’ families are economically disadvantaged, reading scores vary substantially and are generally higher for students from non-economically disadvantaged families. Some states seem to be doing better than others at teaching reading to economically disadvantaged students.
Do you see what we see when looking at the left panel in Figure 4?
Both Mississippi and Louisiana are surpassing the nation as a whole
when it comes to the performance of students
from economically disadvantaged families.
- Mississippi has the highest average Grade 4 reading score (212) among economically disadvantaged students, 8 points higher than Massachusetts (204) and 10 points higher than the nation (202).
- Both the 10th and 25th percentiles for economically disadvantaged students are higher in Mississippi than in Massachusetts by 13 points even though, proportionally, almost twice as many students are economically disadvantaged in Mississippi (75%) than Massachusetts (40%).
- Mississippi students surpass the nation at each percentile point in the score distribution by 6 to 15 score points, except for the very highest scoring students (at the 90th percentile), with the largest differences at the left end of the distribution.
- With 73% of its students economically disadvantaged, Louisiana consistently outscores the nation by about 7 points at each percentile point except the very lowest.
Looking at the right panel of Figure 4, which shows score distributions for non-economically disadvantaged students, led us to the following observations and a probing question.
- All three states have higher average scores than the nation overall by 7 to 10 points.
- Looking at the left side of the distributions, both Mississippi and Massachusetts have notably higher 10th and 25th percentiles than the nation overall—by 10 to 20 points.
- The largest distinction is Mississippi’s 10th percentile, which is 20-points higher than the nation.
- What supports might these states be providing to non-economically disadvantaged students with the most needs?
Figure 5. Distribution plots of grade 8 NAEP reading scores broken out by family economic status. Average grade 8 reading scores by family economic status for the nation and each state are shown in white. Dashed vertical lines indicate the percentile points for the national distribution.
Let’s move to NAEP’s 8th grade data for our three states to see if the trends hold. Breaking out scores by family economic status, the score distribution plots in the left panel in Figure 5 show that Mississippi and Louisiana perform on par with Massachusetts among economically disadvantaged students in Grade 8. All three states outperform the nation overall for the left to middle portions of the distributions.
However, for students who are not economically disadvantaged, Mississippi has room for improvement relative to other states and the nation. Looking at the right panel in Figure 5, we make the following observations:
- Compared to both Massachusetts and Louisiana, Mississippi scores lower across the distribution both for higher- and lower-scoring students.
- For instance, in Mississippi, the 25th percentile is 243, which is 15 points lower than Massachusetts’s 25th percentile of 258 and 10 points lower than Louisiana’s at 253.
- This result for Mississippi’s non-economically disadvantaged 8th grade students is strikingly different from the 4th grade findings, suggesting reading interventions may need to be tailored to student needs as they develop more advanced reading skills in upper grades.
- In contrast, for non-economically disadvantaged students, Louisiana performs more similarly to Massachusetts—often outscoring Mississippi by as many as 10 score points.
By using a common assessment program, NAEP data can provide valuable, detailed comparisons of student performance between states.
Average NAEP scores show that Massachusetts students outperform students in Louisiana, Mississippi, and the nation overall, but stopping at averages would lead viewers to miss higher achievement levels in Mississippi and Louisiana for lower scoring students. Specifically, Mississippi, and sometimes Louisiana, are outperforming Massachusetts and/or the nation overall in Grade 4 reading for
- lower scoring students across the state;
- economically disadvantaged students, particularly those scoring at or below the average score; and
- non-economically disadvantaged students, in particular those scoring in the bottom 10th and 25th percentiles.
- Both Louisiana and Mississippi are mostly keeping pace with Massachusetts in grade 8 reading scores, especially among economically disadvantaged students. There is room for improvement in Mississippi’s grade 8 reading scores, particularly among non-economically disadvantaged students.
Our hope is that state education leaders can use these results as a jumping
off point to try to learn more about what policies and practices led
Louisiana and Mississippi to outperform the nation or other states,
especially when it comes to improving reading outcomes for lower scoring
students and economically disadvantaged students.
Further exploration of student performance by the teacher and school characteristics available in the NAEP Data Explorer could show you which teacher and school practices may be related to higher student performance for these states or your own state. Click here to begin your own exploration of the data.
Marisol Kevelson, Katherine E. Castellano, and Emily Kerzabi work at the ETS Research Institute. Mari is an associate research scientist; Katherine is a principal research scientist; Emily is a research project manager. Eli Holder is the founder of 3iap.