Why are Graduation Rates Increasing?

This post is the first of a series on student achievement. 

The National Center for Education Statistics (NCES) summarizes graduation rates for first-time full-time undergraduates here. Overall rates have increased from 34% to 47% between start years of 1996 and 2014-an average increase of .7% per year. Why is this?

Educated Guesses

"Why" questions are a trap, like a free trial subscription to the Wall Street Journal. See The Book of Why for an accessible discussion of the history of causal thinking and details of path-diagram method that Judea Pearl developed. One problem with finding causal explanations is that it's too easy to imagine a cause from some effect we'd like to explain. In logic this is called abductive reasoning, and historically this has led to a lot of problems, like "the barn burned down because Ahab used witchcraft." However, abduction is useful for formulating guesses about causes, as long as we're willing to be wrong about all of them. In that spirit, here are some possible reasons for the graduation rate increases that are empirically testable.

1. Colleges are making it easier to graduate because:
   
1. Requirements are more lenient
2. Student support has increased
    
2. Student population characteristics are changing in favor of those more likely to graduate
   
   
3. Student-institution matching has improved
   

The first of these is straightforward to understand. The second suggests a "selection effect" of the type that tripped up the 1983 report A Nation at Risk (see chapter 7 of Cathy O'Neil's book). Suppose the group of students who are least likely to graduate college become discouraged over time, perhaps because of rising costs and diminishing public confidence, so on average a smaller fraction of them choose to attend college each year. That self-selection out of the applicant pools would be expected to result in higher graduation rates.

The third hypothesis is even more subtle. On average, students are applying to ever more colleges each year, casting the net more widely as it were. This suggests that the academic and financial match between students and institutions might be improving--a kind of free market hypothesis. If so, this effect might be discernible in the data as a connection between higher application rates and higher retention or graduation rates.

But let's start with the first hypothesis on the list, because there has already been some good work done to ask the question. 

Grade Inflation

I first saw this research mentioned in Inside Higher Ed: The Grade-Inflation Completion Connection. The published paper is here, and the citation is

Denning, J., Eide, E., & Warnick, M., Mumford, K., Patterson, R. & Warnick, M. (2022). 

Why have college completion rates increased? American Economic Journal: Applied 

Economics, 14(3): 1–29 https://doi.org/10.1257/app.202005

The abstract reads

We document that college completion rates have increased since the 1990s, after declining in the 1970s and 1980s. We find that most of the increase in graduation rates can be explained by grade inflation and that other factors, such as changing student characteristics and institutional resources, play little or no role. This is because GPA strongly predicts graduation, and GPAs have been rising since the 1990s. This finding holds in national survey data and in records from nine large public universities. We also find that at a public liberal arts college grades increased, holding performance on identical exams fixed.

The data used in this paper comes from student-level data from national surveys (among other sources), but is limited to the period 1988-2002 for some data and up to 2010 for other data. 

 

Note that the abstract signals that the authors have considered selection effects and increasing student support (two of the hypothesized causes on my list), but find the most evidence for GPA increase as the largest identifiable cause of increased graduation rates. 

 

In the discussion, the authors point to other factors not on my list (emphasis added):

We discuss relevant trends that could affect college graduation, such as the college wage premium, enrollment, student preparation, study time, employment during college, price, state support for higher education, and initial college attended. The trends in these variables would predict declining college graduation rates almost uniformly. 

In other words, given what we see, we should expect to see graduation rates decreasing. Instead, "changes in first-year GPA account for 95 percent of the change in graduation rates" and these changes in GPA can't be accounted for by student characteristics like better preparation or college test scores (i.e. measured learning isn't increasing with grades). In common parlance, grade inflation in the first year of college is making it easier to progress and graduate. Additionally, the largest effect is seen in improvements to the low end of the GPA range:

In each sample, and in each cohort, the change in the probability of graduation is largest for GPAs between 1.0 and 2.5. That is, improvements in GPAs in that range correlate with meaningful increases in graduation, whereas GPAs above or below that range do not change the probability of graduation as much.

An analysis of (quasi-) standardized tests shows that scores didn't increase along with GPA, which supports the grade inflation conclusion. 

The authors published an overview for a general audience in The Chronicle as "The Grade Inflation Conversation We're Not Having." They explain the hypothesized mechanism for the graduation rate increases:

It is easy to see how a focus on graduation could affect grade inflation. Imagine that a college wants to increase its graduation rate. An institution under such pressure has several options at hand. It could maintain grading standards and help students via tutoring or other student-success programs. It could change who is admitted (if it is a selective school). These are costly changes to make, and any particular program may not work.

Alternatively, the college could relax its grading standards and suggest or accept that what used to be D-level work is now worth a C. Relaxing grading standards has the advantage of providing no direct cost to the university, the professor, or the student.

This stark appraisal of the economics of higher ed says it all: there's a market for degrees, not so much for learning. Although there are important caveats, this idea echos Brian Caplan's tune.

Recent Graduation Rates

The period covered in the study ends over twenty years ago, but the NCES data shows that graduation rates continue to increase. To try to understand trend in the context of Denning, et al., I used IPEDS data on four-year undergraduate institutions, and compared a selectivity measure to four-year graduation rates.

In the figure, AppsPer is applicants per enrolled student, where higher levels mean more demand for four-year undergraduate programs at a university. Since applicants increasingly apply to more and more schools, I binned this value within each year to be consistent, using quintiles, so AppsPer 5 comprises the institutions with the largest number of applicants for each enrolled student for that year. Larger pools of applicants allow colleges to be more selective. 

As expected, more selective schools have higher graduation rates across the time period. While all selectivity ranges saw increased graduation rates, this is most dramatic in the least-selective institutions. The pattern is similar if we use standardized test scores instead of applicants per student. These patterns are consistent with the idea that the first year of college is easier to pass for the least-prepared students. That can't be the whole story, but it seems likely to be part of it. And it seems likely that the grade effects found in the paper have continued, and perhaps have accelerated.

Conclusions

I take from this study a couple of useful points. The first year of college is really important. We already know that first year retention is important, and at my university the most important predictor of retention is grades. That relationship is non-linear and has a discontinuity post-covid that penalizes low grades even more after the pandemic. We surmise that covid interfered with high school preparation of math and writing skills, which affect the least-prepared students the most.

This study shows how grading and learning can diverge, and there is a tension between two things we care about:

1. Less grading rigor increases retention and graduation
2. More grading rigor is associated with more learning
   

This tension puts access and accountability on a collision course. Before jumping to conclusions, however, I recommend reading at least the conclusion section of the Denning, et al paper. The authors are careful to outline various implications, for example without making an overall judgment that the GPA trends lead to bad outcomes for students. I'll discuss a few other current research articles to dig into these questions in the next few posts.