Duval [County public school] students passed 80 percent of their AP courses last year with a "C" or better. But only 23 percent of the national AP exams, taken near the end of those courses, were passed.
The national exam pass rate for public schools was 56 percent.
In other words, students successfully complete a course that is essentially a preparation for a standardized test, and then fail said test. The College Board, which reviewed the results, blames the effect on under-prepared students taking the courses combined with inexperienced teachers. When I read this, I thought it was perhaps a good opportunity to look for evidence of a phenomenon I've said should exist: that standardized tests are more valid for low-complexity subjects of study. Here, complexity means in the computational sense (search for the word in my blog for lots more on the subject). If we assumed all things equal (dubious, but I have no choice) then preparation courses for lower complexity subjects ought to be more effective than for higher complexity subjects. This would manifest itself in the correlation between passing tests and passing AP exams. This is all possible, because the statistics for the school district in question are posted online.
In pure complexity terms, math is low complexity and languages are higher complexity. This is easy to see--math is a foreign language with little new vocabulary and a few rules. Spoken languages have massive vocabulary and many, often arbitrary-seeming, rules of grammar. So if my theory is any good, it ought to be the case that math courses can prepare a student better than language courses for a standardized test, all else being equal. Also, the overlap in students between the two courses is probably pretty good since college-bound seniors will be taking both a foreign language and math. Of course, even learning a foreign language is mostly committing deductive processes to memory, and hence of not the highest complexity (inductive processes would be). So this is a contest between low complexity and, shall we say, medium complexity.
Here are the results:
I debated whether or not to include both calculus courses (clearly, more advanced students are in the BC section). I also assumed that the three languages are equally complex, although in practice Spanish dominates. If a student passed a calculus course, he or she had a 48% chance of passing the AP subject test. For languages (the more complex subject), only 39% of those who passed the course went on to succeed on the exam.
Does this prove anything? Not really--there are too many uncontrolled variables. But it's still fun to push this as far as it can go. If the complexity to difficulty relationship holds, we would expect that the subject with the worst test/course pass ratio would be the most complex subject. Of course, sample size plays a role, so let's agree (before I look at the numbers) that there had to be at least N=50 to qualify. For all tests combined, the average test/course ratio was 29%. Anything lower than that would be lower than average preparation for the test (and higher complexity maybe). The least effective (or most complex) course was a three-way tie, with a 16% conditional probability of passing the AP test, given that the course had been passed. The three subjects were World History, Human Geography, and Micro Economics. These each had enrollments in the hundreds and thousands.
Are these subjects the hardest to test because of complexity? It's easy to guess that the first two might be, cluttered with endless facts and fuzzy theories. Micro Economics is much more like chemistry or physics, one would think. Chemistry scored a low 22%, but Physics B was 58%.
This was fun, but it's hard to really make the case that complexity is the driving force here. It does give me some ideas, however, about comparing difficulty (course pass rate) versus complexity (course to test pass ratio). Meanwhile, I still have the placement test project to try out...
[Update: here's an interesting article about the effectiveness of the calculus AP test]
No comments:
Post a Comment