Higher Profile for Testing Proposal

Currently the most emailed New York Times article (subscription may be required) is entitled "Panel Explores Standard Tests for Colleges." A quote:

A higher education commission named by the Bush administration is examining whether standardized testing should be expanded into universities and colleges to prove that students are learning and to allow easier comparisons on quality.

This is of course the Commission on the Future of Higher Education, chaired by Charles Miller. Oddly, he's quoted as saying "There is no way you can mandate a single set of tests, to have a federalist higher education system." I agree completely with that statement, but I'm not sure how they propose to achieve their stated goal of comparing institutions if each uses a different metric.

The article quotes Leon Botstein and David L. Warren staking out a reasonable position from the perspective of those actually doing the educating, opposing "a single, national, high-stakes, one-size-fits-all, uber-outcome exam."

Predictably, standardized test fans like the proposal. Here's a sample:

Any honest look at the new adult literacy level data for recent college grads leaves you very queasy. And the racial gaps are unconscionable. So doing something on the assessment side is probably important. The question is what and when.

And there's a connection to the CLA, which I've written about before here.

Mr. Miller, in his recent memorandum and in the interview, pointed to the recently developed Collegiate Learning Assessment test as a breakthrough. The exam, developed by the Council for Aid to Education, a former division of the RAND Corporation, asks students to write essays and solve complex problems.

The central problem is that in order for a uniform exam to have high reliability the scoring has to be uniform. In the best case this is done by a computer. Unfortunately, this algorithmic 'right or wrong' scoring is not aligned with the typical goal of higher education to deliver higher-order cognitive skills. What is 2+2? Correct answers include 4, IV, 100 (binary), vier (German), and 1 (modulo 3). Standardized testing by its very nature ignores any component of imagination that makes its way into the answer if that imagination introduces unwanted variability (which seems terribly likely).

See my post on validity for more. The National Council of Teachers of English has a very well-considered set of standards for judging writing. There's no way that they translate into standardized testing, however, which means they would be ruled out in the proposal under discussion.

Critical thinking is one of the abilities that some standardized tests purport to assess. Unfortunately, the decision seems to be driven by the economics of test-giving (it's easy to score bubble-sheets), and a misplaced importance on reliability. One would hope that the Commission will apply some critical thinking to this problem.