Wednesday, September 02, 2009

Zza's Best Liberal Arts Schools

US News college rankings get a lot of press, not just because of the rankings themselves. Critics rave about how arbitrary the scores are. Heck, I've done it myself. Just to show how easy it is, I decided to to create my own college rankings and become rich and famous just like them. All the data you need is freely available on the IPEDS Executive Peer Tool. If you want a numerical rating based on the best quality data you're likely to get it, that's the place to start.

I decided to look at mid-sized liberal arts institutions, so I picked Davidson College as my focus institution and then used the filters to select all similar schools. There were 171 of them. Unfortunately, the reporter will only spit out data on 100 at a time, so I had to break it into two groups. But never mind that. A sample from the data menu is shown below.

Note that the data is a little out of date, but it's the most recent in the system.

To keep things simple, I decided to see where you're likely to get your money's worth as a student. I chose several variables to look at, including headcount, average tuition, average institutional aid award, and instructional costs per FTE. In the end, the ranking formula I chose was a simple one:
Value = Benefit/Cost = (amt spent on instruction/FTE) / (tuition - inst. aid)
Here, (tuition - institutional aid) gives us the actual average cost to attend. Dividing this into the amount spent on instruction per FTE should give us an idea of how much quality one gets for one's dollars.

Without further ado, here is the list for all institutions with Value > 1.0 (rounded).

The ratios are quite striking. For every dollar you spend on tuition at Williams College, you're getting over five times as much in return, just for instructional costs! No other institution even comes close to that.

I also looked at how much endowment income per student was in relation to how much institutional aid was awarded. I assumed a nominal 5% return on investment. These numbers give you an idea of how much of the endowment return is spent on financial aid: the ratio of endowment return per FTE divided by average aid per FTE gives you an idea of how the school uses its endowment. For the top five schools on the list above, this is more than 100%, meaning that each of them has endowment draw left over after "paying out" institutional aid. This "Endowment Power" index is labeled Draw/Aid below.

I would take this to confirm these five as being a good value for your money considering the professorial talent on hand and the extra money that goes to fund administration and other wonderful things. Of course, you have to get admitted first...

In case you're wondering, the US News list has these top five liberal arts colleges:
  1. Williams College
  2. Amherst College
  3. Swarthmore College
  4. Middlebury College
  5. Wellesley College (tied for fourth, actually)
Pomona is tied for sixth. Middlebury College isn't on my list because it doesn't have tuition data listed in the IPEDS dataset I got. I dug around on their website just now and found the current tuition to plug in, but the value quotient only comes to 1.0 (rounded), so they wouldn't have made my list anyway.

It's interesting to graph the Value, as I defined it against the Endowment Power in a scatterplot, shown below.

The outlier on the left is Brigham Young University-Hawaii. The cluster above and to the right of the main mass is our top five. They really are in a category of their own. Zooming in is interesting: it shows further differentiation, as if there's a power law at work here, but I haven't checked that.

If you'd like to play with the data yourself, you can download it in tab-delimited text format here. The following institutions had missing data of one sort or another, and are not included in the set:Bates College, Colby College,Connecticut College, Lindsey Wilson College, Middlebury College, Union College,St Lawrence University,Whitman College.

I guess I have to take back any snarky comments about US News I may have made. It looks like their top five rankings for liberal arts schools are pretty sound. This isn't to say the rest of the listings aren't a bunch of hooey--they may be, but I haven't seen it yet. I was all prepared to argue (see my previous few posts) that they were generating and selling estimator error, but this is a case of a nice theory encountering actual facts.

A whole other line of thought involves the actual costs of these schools. Notice how low the tuition - aid averages are? And this is just institutional aid, or discounting as it's sometimes called. The prices on that list don't seem to jibe with all the rhetoric about inflation of higher ed costs. But I may be premature...that's a topic for another time.

You can head over to IPEDS and roll your own metrics.

Update: A commenter (see below) caught the fact that not all students receive institutional aid, which means the conclusions only apply to students who get that aid. Some stats on that are given in my comments below. This fact would also skew the ratio I called endowment power, since denominators are different in aid/aid_students and return/all_students.

I also fixed a typo.

Update 2: I posted another list for full-pay students here.

10 comments:

  1. Anonymous8:04 AM

    To calculate average aid are you using the number receiving aid as the denominator? That is often very deceiving as the average usually goes up as the number receiving aid goes down among selective colleges. A better way to do it is to spread the aid expenditure out over the entire student body to get a true discount and also a clearer picture of how aid affects the ability to spend.

    ReplyDelete
  2. Thanks for the comment. The average institutional aid per student comes right out of IPEDS, but it does indeed only cover students who receive institutional aid--your point is a good one. I'll go back and grab the percent of students getting institutional awards to see what difference it makes.

    ReplyDelete
  3. Here are the percentages for Williams College and its auto-selected peer group. The numbers are percentage of students who receive institutional aid. Rescaling to average aid over all students gives a better 'average' picture, but no one student simultaneously receives aid and doesn't receive aid, so the average isn't very meaningful. Would be better to have two charts, probably, one for these 'merit' students and another for the rest.

    ID Name
    168342 Williams College 46
    128498 Albertus Magnus College 49
    210571 Albright College 87
    143084 Augustana College 98
    189088 Bard College 64
    189097 Barnard College 45
    202523 Denison University 94
    150400 DePauw University 98
    212009 Dickinson College 51
    184348 Drew University 99
    133492 Eckerd College 96
    212577 Franklin and Marshall College 60
    182795 Franklin Pierce University 89
    145646 Illinois Wesleyan University 93
    213385 Lafayette College 57
    166850 Merrimack College 93
    166939 Mount Holyoke College 61
    204185 Mount Union College 99
    183239 Saint Anselm College 87
    231059 Saint Michaels College 96
    239716 Saint Norbert College 94
    141060 Spelman College 49
    195216 St Lawrence University 78
    167996 Stonehill College 81
    130590 Trinity College 38
    196866 Union College 58
    197133 Vassar College 48
    234207 Washington and Lee University 41
    168218 Wellesley College 54
    206525 Wittenberg University 98

    ReplyDelete
  4. Anonymous6:26 PM

    Applying a correction for the percent on aid is certainly better than not applying one but, even so, this is not sufficient to resurrect a fundamentally flawed analysis. "Average grant" may or may not be an indication of the cost of an institution to a student from a family in any particular income bracket. It is also a measure of the distribution of family incomes in the financial aid population at the institution...something that is not revealed by IPEDS data and not any indication of "value" or "quality". To illustrate with an overly simplified example: Let's say that Student A from a $30,000 income family gets $40,000 of grant assistance, and Student B from a $150,000 family gets $5,000 of grant. If an institution has two Student A's, their "average grant" will be $30,000; if it has one of each the "average grant" will be $17,500, and if it has two Student B's the "average grant" will be $5,000. Is there a difference in "cost" between these two institutions? Of course not! Without information about the income distribution of students at different institutions, "average grant" data are meaningless.

    ReplyDelete
  5. Yes, averages suck for most things. That's what happens when you data compress a rich data source into a few bits. I avoid averages when I can for that reason. But on the other hand, we don't have access to all the admissions matrices for the institutions on the list. The averages do say something about the way the college or university is run, and give an idea of how much aid is in the offing. The only way to know the specifics for a particular student is to actually apply.

    In your examples you focus on income, but most institutional aid is merit-based. And there is a difference in cost in the examples: we can argue about utility, but the cost of T-5K is different from T-30K. Without revenue from another source, like a fat endowment, many institutions would like a bunch of the the first, so they can add N*(T-5K) to the net revenue line.

    ReplyDelete
  6. Anonymous12:46 PM

    I noticed that you used tuition figures from 2007-08 and financial aid data from 2006-07. Was this an oversight, or is there justification for doing so?

    ReplyDelete
  7. That was an oversight. Thanks for pointing it out.

    ReplyDelete
  8. Anonymous4:08 PM

    Financial aid at more than half of the institutions listed in the original post is awarded almost entirely on a need basis and in most of those cases it is 100% need-based. This is especially true at the top of the list. So, in truth, the data for those institutions may say something about the economic profile of the students who enroll but practically nothing about how "generous" the institutions are or what the relative costs are for similar students at different institutions.

    ReplyDelete
  9. Aid may be need-based, but in order to get admitted at all you have to demonstrate merit. Williams, for example admitted 18.5% of the class of 2012 apps. With 75% of the students (2008) having SAT scores of over 1300, they are sought-after and able to command discounts at about any institution. So need-based aid, given a very high bar of presumed talent, is a better description. From their website: http://www.williams.edu/home/fast_facts/

    Class of 2012 Financial Aid Statistics
    Percent receiving Williams aid 50%
    Average financial aid award $40,656
    Range of awards $3,000 - $55,792
    Range of family income of students assisted $0 - $218,000

    I'm not sure what the metric for generosity is. The one I used was simply average instructional value per net tuition dollar. Nor is there enough information on IPEDS to ask what is the best value for this particular student. But the averages can give you an idea of where to apply to. IF you can get in, Williams delivers a big return on your investment, considering only instructional costs of course. That's a combination of billion-plus dollar endowment plus policies that put a lot of money in the classroom and an intelligent financial aid program. Another place where this last shows up is in the diversity statistics, which you can also find on the information page above.

    ReplyDelete
  10. Anonymous1:05 PM

    Yes, getting admitted is the first hurdle. That is a given. But, as you acknowledge, once admitted the averages that you can get from IPEDS data provide no information about the best value for a particular student. If you believe that a student choosing Willliams will necessarily experience a lower net cost than (the same student) choosing... say...Oberlin, then you are completely misunderstanding how financial aid works. IPEDS data tell you absolutely nothing about relative cost at different institutions for a particular student. In fact, if Oberlin gives some "merit aid" and Williams gives only need-based aid, it is entirely possible that a student will experience lower net cost at Oberlin. You are giving bad advice if you suggest that families should interpret these averages a any indication of what their costs will be. The only way to find out is to apply, get admitted, and apply for financial aid.

    ReplyDelete