This year we changed our course evaluation form from a very long list of management questions ("Did the instructor meet office hours regularly?") to a short one focused on learning outcomes. The report below is from my fall 2010 Calculus II class.
That's the whole form, except for free-form written responses. As a first look at the meaningfulness of results, I looked at the correlation between responses for each item, and got a surprise. To wit: while items 9,10, and 11 are tightly correlated, as shown on the matrix below,
and furthermore, items 12 and 13 are correlated at .88,
these two summative questions (12 and 13) do not correlate well at all with any of the other items, including 9, 10, and 11, which ask if the course was interesting, enjoyable, and did you learn. Do students not associate those things with their summative evaluations, or is there some other register that is engaged (i.e. they perceive that the last two questions are more personal, about the instructor instead of about themselves)? On the other hand, the very low correlation between item 2 (How much effort did you put into the course?) and the overall evaluation seems to be a good sign, implying that students differentiate between their own performance and that of the instructor.
