Kill the Darlings

The title comes from standard advice to writers; for origins see this Slate article. It occurred to me during my daily scan of Andrew Gelman's blog:

[I]n science (or engineering, business decision making, etc.), you have to be willing to give up your favorite ideas.  

The full post is here, and worth reading for the cautionary tales of what can happen when we cling too long to beautiful ideas in the face of ugly facts to the contrary.

Transparency Act

In other news, draft legislation on college transparency can be found here. It proposes a student-unit data tracking system (which would overturn current law), with the goal of making outcomes transparent. Here, outcomes include what happens in college (retention, graduation), but we already have those, so the new addition is the effects on graduates. From page 21:

If I read it correctly, the list of categories to be disaggregated comprises:

• (I)  Enrollment  status  as  a  first-time  student,  recent  transfer  student, or other non-first-time student. 
• (II) Attendance intensity, whether full-time or part-time.
• (III)  Credential-seeking  status, by credential level. 
• (IV) Race or ethnicity. 
• (V) Age intervals. 
• (VI) Gender. 
• (VII) Program of study (as applicable). 
• (VIII) Military or veteran benefit  status 
• (IX) Status as a distance education student, whether  exclusively or partially enrolled in  distance education. 
• (X) Federal Pell Grant and Federal loan recipient status

 To make maximum use of this (as yet hypothetical) data, it's interesting to think of a system like the net tuition calculator for predicting future outcomes. You'd enter your information, including prospective field of study, and receive likelihood of graduation, average predicted salary, and so on. The most important predictor missing from the list is a measure of past academic performance, like high school grades and standardized test scores. For collegiate outcomes, it would be important to include GPA bands, I believe.