Scorecard Salaries

As announced in InsideHigherEd this morning, the College Scorecard is now reporting financial information for graduates by institution and program, when there are enough samples. I used the opportunity to play what-if.

Back in ancient times, when I was an undergraduate studying math, I decided I'd need another major in order to find a job (don't ask me why I thought that), so I picked up Electrical Engineering. Eventually stayed with math, but I completed all but about three courses from the EE degree.

Here's what the Scorecard reports for recent graduates.

Yeah, I'd probably be retired by now.

Data Completeness

The data for the Scorecard comes only from Title-IV eligible students, and there is the question about how representative that is of the whole student population. A couple of years ago I compared the downloadable data from the Scorecard to the data from the Equality of Opportunity Project, which includes all students, not just Title IV. At the institution level, I got a >.90 correlation between the two. Here's a scatterplot.

This was part of a larger project to see if I could account for variation in graduate salaries by taking into account the distribution of subject that graduates had. You can find the data and code here. 

Real Outcomes

One of the best assessments of college is the job and the paycheck. The chart above tells the tale (taken from here). Another table from the Wall Street Journal gives data by major, which is very interesting (hint: liberal arts comes out looking good)--see this post for details and comments. I argue there that the government is in the best position to do detailed studies by institution if they really are interested in learning outcomes. I had an opinion piece published about that idea in University Business a while back. Maybe the new administration would be more receptive to such an idea.

Another measure of success of higher education is the percentage of the population with degrees.
These are all solid, usable statistics with no statistical goo added. Notice that there aren't any averages in sight (unless you consider a percent to be an average over a binary variable).

This contrasts with the generally opaque assessments of learning outcomes. Part of the allure of standardized testing is that at least the outcome seems to be clear: it arrives in the form of a crisp number like the ones above. These can be turned into pretty graphs. The problem there is whether or not they actually mean anything. I've argued that they can, if the complexity of what's being tested is appropriately low. Testing calculus (low complexity) may work, but testing critical thinking (high complexity) probably won't.

The Wealth of Majors

I've argued before that average graduate salaries would be interesting to use in program and institutional assessment. I've thought that the government ought to report out these average salaries for graduates of an institution to use as an outcomes measure. They have all that data anyway--why not use it for something? I've even made this pitch to people who work in the Department of Education, but I invariably get a stony response.

There's a report shown in the Wall Street Journal showing data collected by a private company to accomplish the same thing. Take a look and see if this isn't useful information. I wish that for comparison's sake, they'd included people who never finished college, or never finished high school.

Disambiguating cause and effect is difficult, of course. Impossible with the data shown. We don't know if MIT graduates earn more than SIU graduates because the school taught them better or the students were better to begin with, or if perhaps they settled in locations nearer their schools, which have different economies. It may say more about what kinds of people attend what kinds of institutions and major in what kinds of subjects than it does about the effectiveness of a program of study in the eventual production of wealth. Still, this kind of information is useful for parents and students considering their options. The fact that median salaries for philosophy majors is slightly more than that of marketing majors makes me happy.

If the government really were to do such a study, using tax records, financial aid data, and clearinghouse data, some clever person might be able to find a control group in order to untangle causes and effects. A good salary shouldn't be the sole end of education (see this book on that topic), but it's an important factor, and undoubtably useful.