Test Scores and Parent Income: How SchoolSparrow’s Rating System Works

SchoolSparrow’s K-12 public school rating system provides more truth about school quality. We measure the impact that a school community is having on student’s standardized test performance.

The other ratings you see everywhere are not communicating what parents think they are communicating.

Test Scores Are Highly Correlated With Parental Education/Income

Parents searching for homes care about school quality, and they can’t possible visit every school in a city to figure out which school is the best for their kids. So parents use other methods to whittle down the options, and one of those methods is a cutoff on the school score as published by GreatSchools. GreatSchools scores are published on all the major real estate search portals, and their ratings are significantly influenced by standardized test results. But research has shown over and over again that student test scores are 70-80% attributed to parent education/income. That’s not to say that schools don’t influence test scores, they certainly do. But the school’s impact pales in comparison to what is happening at home.

As a result, the GreatSchools rating describes more about parental education/income than the quality of the school itself. A school comprised of students from affluent families with high test scores, doesn’t necessarily equate to high-quality teaching. And the inverse is true as well. A school comprised of families from socio-economically diverse or moderate to low incomes with lower test scores doesn’t necessarily equate to low-quality teaching.

This piece by chalkbeat helps illustrate the issue.

A New Rating System

That’s why SchoolSparrow created a school rating system that takes into account each school’s socio-economic profile.

Imagine a two schools, both with the same standardized test scores and both with a rating, of say, a 7/10. Are these good schools? Realtors report that parents want a 7 rating or better, particularly if they are moving to a new city, or they will pass over the school and the homes and neighborhood the school serves. A 7/10 passes muster with these parents.

But then it’s revealed that one school is located in a homogenous suburb with mostly high income students. While the other is in a socio-economic diverse area near the major central business district.

Now what can we say about these schools? Well, the test scores for the school in the high-income suburb might be in the bottom 10% of schools in high income areas. Is that a good school? Maybe, maybe not.

And the test scores for the school in the city might be in the top 90% of schools with similar income profiles. Is this a good school? Absolutely. What is this school doing right?

And more often than not, the high-income school will be racially homogenous, and the urban school will have more socio-economic and racial diversity. The cognitive benefits to children in diverse schools is well documented. They include better teamwork and decision making skills, social-emotional development, more acceptance and understanding of other cultures, and possibly even higher IQ’s. These are the kids we want leading the next generation.

Our rating system has very different scores for these two schools because we account for each school’s socio-economic profile. Without this context, comparing two school’s test scores is actually meaningless. You might as well compare the size of parents wallets.

SchoolSparrow’s Algorithm

Our rating system is a multi-variable non-linear regression that predicts the average or expected test score for school’s with similar demographics. Our algorithm has been validated by a data scientist out of the University of Chicago. For every school in America, we calculate the school’s average expected test score. Then the school’s rating is based on where the school’s actual test score falls in comparison with the average. In this way, we are measuring the impact that the school community is having on student performance, while controlling for socio-economic factors.

The graph below is a simplistic representation of how our rating system works. This graph is a single-variable linear regression. But it provides a visual example that shows how our rating system tells more truth about school quality.

Here we’ve plotted all schools in the Boston Metro Area. The x-axis represents parent income in the form of the percentage of kids at each school who are considered economically disadvantaged. As the percentage increases, the average parent income is falling. The y-axis represents test scores in the form of the percentage of kids at each school who were deemed proficient on the Reading/Language Arts section of the standardized test.

This data has a trend-line. The trend-line has an equation that allows a calculation: given a school’s percentage of economically disadvantaged kids we can calculate the average expected score for schools with similar economic profiles. We call this the school’s expected score.

By subtracting the expected score from the actual score, we get a measure of the impact the school community is having on test scores. On our system, schools are ranked by this difference.

We’ve highlighted the trendline for Boston in the graph above. The trendline has an R-Squared value which is an indicator of how correlated parent income is to test scores. An R-Squared of 0.2 means two things have some correlation. The R-Square in this graph is 0.72, which mirrors the research that test scores are 70-80% attributed to parent income.

Hill Elementary

We’ve highlighted Hill Elementary on the graph above. This school has a low GreatSchools rating of 6/10 because the test scores are about equal to the state average. The 6/10 tells us more about parent’s incomes: they are probably about equal to the state average. But that doesn’t tell us much about the quality of instruction at Hill Elementary.

On our system, this same school has a high rating because the teachers are having a large impact on student performance. We know this because of the large gap between the expected score given the socio-economic profile of the school, and the school’s actual performance. This large gap is a result of the impact of teachers at Hill Elementary.

We’ve interviewed school leaders and teachers at schools like Hill Elementary, where there is a low GreatSchools score but our rating is very high. We call these schools Out-Performers.

We found several themes. Often there is a strong principal that has cultivated a positive culture, high teacher morale, and partnerships with at least 3 community organizations that provide extra services for students at the school. Students thrive this these environments.

Additional Commentary

It goes without saying that selecting a school is a very personal decision for every family. Parent’s should have access to school data. The unfortunate thing is that America’s de facto school rating system isn’t being clear about the powerful connection between parent income and test scores. The ratings don’t communicate what parents think they are communicating.

There are families living and moving to neighborhoods across the entire spectrum of socio-economic status. From affluent families with lots of choices to families experiencing poverty with few options when it comes to schools.

For the affluent family considering schools in high-income areas, our ratings will reveal schools that, although they have high ratings, are actually falling short of expectations.

Similarly for families living in lower-income areas, our ratings might reveal that the school two blocks over is actually out-performing your kids current school.

Today’s rating system obscures these distinctions because schools where parents have high incomes tend to get high ratings, and schools where parents have diverse, or moderate to low incomes tend to get low ratings. These ratings don’t tell the truth about which schools are having positive impacts on student performance.

A Final Word of Caution

We actually don’t think giving a rating to a school is a good practice. Especially when the rating is based on one small factor that describes a school, like a test score. Schools are complex ecosystems and one number cannot possibly encapsulate all that makes a child lover her school.

But a ubiquitous rating system is doing just that, but in a way that misleads parents, because parents trust that these ratings are describing school quality. When in fact, these ratings are a better representation of the socio-economic status of parents. Today’s de facto school rating system is not communicating the powerful relationship between test scores and parent income.

If you are going to evaluate test scores, then our system will reveal more truth about which schools are impacting student performance.

But just because a school has a low rating on our system doesn’t necessarily mean it’s a problem school. There are so many factors at play, and we encourage families to visit as many schools as possible to find the right fit for their family.