Friday, February 19, 2010

Middle Income Students

The College Board has a new report: "Paying for College: Students from Middle-Income Backgrounds."

"Families at the upper end of the broad middle-income range have seen their incomes increase slightly, while those at the lower end of this range have seen their incomes decline."

This is a very short, bullet-point type report, and worth taking a look at. Particularly if you're trying to figure out financial aid strategies.

Monday, February 08, 2010

Validity for Gen Ed Tests

I came across a relatively new page on validity studies on the Voluntary System of Accountability (VSA) site. The studies focus on the CAAP, CLA, and MAPP, which are all aimed at critical thinking and writing to one degree or another. There are several reports to be found there, including an executive summary. In Interpretation of "Findings of the Test Validity Study Conducted for the Voluntary System of Accountability," we find that the correlation between the tests in question, when compared at the school level ranged from .75 to . 87. Here, "School-level reliability refers to score consistency (i.e., a school receiving a similar mean score regardless of the sample of students taking the test)."

Correlations between test sections is show in the table below, from page 6 of the report.

This is presented as evidence of construct validity. I don't see any evidence that they did a confirmatory factor analysis to check the dimensions.

The researchers tried to tease out effect size by looking at the difference between freshmen and seniors, but had to jiggle the statistics to account for the fact that the seniors and freshmen were different people, with different characteristics (there was a difference in SAT average, for example). This is not surprising because of survivorship. The actual method for this renorming is not given in this report (maybe it's in one of the others), but there's a graph to illustrate that after fiddling, the seniors look better than freshmen. Except in math.

This finding is interesting to me as a math teacher. I find it to be plausible, since college freshmen would generally have just come out of a several continuous years of math instruction, while college seniors would not.

There's more in the report, including some cautions on page 11, that's worth reading.