Debating the value of SAT scores

A friend sent me a link to a New York Times article about the connection between student SAT scores and graduation rates in the SUNY system. Peter Salins, the author, poses the following question:

[D]o SATs predict graduation rates more accurately than high school grade-point averages? If we look merely at studies that statistically correlate SAT scores and high school grades with graduation rates, we find that, indeed, the two standards are roughly equivalent, meaning that the better that applicants do on either of these indicators the more likely they are to graduate from college. However, since students with high SAT scores tend to have better high school grade-point averages, this data doesn’t tell us which of the indicators — independent of the other — is a better predictor of college success.

He explains that the admissions standards of the SUNY system in the 1990s created a natural test as some institutions raised SAT requirements while others didn't, and high school GPA requirements remained roughly the same. The former schools saw significant increases in graduation rates. He concludes that those who wish to do away with SAT requirements are ignoring important information.

One question that the article doesn't answer is who exactly is graduating? Are the higher SAT scorers the ones who are responsible for the graduation rate increases? One would assume so, but it doesn't take much experience in institutional research to learn that you shouldn't assume such things. I ran our numbers to see what the situation is here. The graph below shows entering classes from 2000 to 2003 by quantized SAT, showing graduation rates. The students who didn't take the SAT (about half) graduated at the same rate as those who did, by the way.


We're a small school, so the two-standard deviation error bars are pretty intimidating, but I think we can see there's no support for the idea that the higher the SAT, the higher the graduation rate for our institution. On the other hand, actual grades earned are a good indicator. That doesn't help much for predictive purposes, but it shows that classroom accomplishments matter.

So for us, SAT is useful as a predictor in conjunction with high school GPA for predicting first year grade averages, but not much more than that. In fact, for our student population (half are first-generation college students) there's a good chance that the SAT underestimates their potential to graduate, leading them to be 'underpriced' in the admissions market compared to schools that put more emphasis on SAT.

Ultimately, it's more important that a student and the institution be a good match than that the student has a high standardized test score. SAT is a very blunt instrument. I think few would dispute that there are correlations with grades and graduation rates that can be useful predictors, but the real question is: is it worth the cost? Are there better ways to match applicants to institutions where they may have a better chance of finding what they want. I'm convinced that in our case there is. Our recent experience with the CIRP survey has convinced me that there are important variables we're not considering when we just look at grades and test scores. Behavior, attitudes, family support, and "cultural capital" are very important. My attitude toward SAT can be summed up in a newly-minted dictionary entry: meh.