Grading and Assessment

The topic of how grades and assessments could be aligned has been mentioned here before (see "Assessing away Grades"). Nils Peterson pointed me to an active discussion on the topic on HASTAC called "Grading 2.0: Evaluation in the Digital Age." There are some good links there, topic questions, and several comments. One of the discussion points asks:

3. Can everything be graded?
- How important is creativity, and how do we deal with subjective concepts in an objective way, in evaluation?

Here, I think we run smack into the problem. It goes like this:

1. Grades have economic consequences for both students and teachers.
2. Because of this, grades have to be defensible during a challenge and review process.
   
3. Because they have to be defensible, grades have to have at least the appearance of objectivity.
4. However: the best assessments should be free from economic influence, and may be subjective (see the whole Assessing the Elephant thing).

This problem only rears its head for complex learning outcomes. If you're teaching multiplication tables, it's not a problem to create objective (maybe even valid and reliable) instruments. What about creativity, however, as posed in the question above? Can we really slice up that concept into "dimensions" and rubrics that capture the essence of what creative genius produces?

It's instructive to see how grades are assigned in the fine arts and performance arts. There simply is a lot of subjectivity. I'm generalizing from limited experience, but I think that the key is the attitudes and methods the assessor uses more than the actual assessment. For example, if I say "your work is all derivative and boring," it's very different from saying "to my taste, this doesn't excite me." The former sounds like an objective statement, and the later is clearly subjective. Students aren't stupid, and they know that there's a difference: dressing up subjectivity as objectivity only irritates them. What I've seen from successful art-type assessments is that the effort put into the work counts for a great deal. Creativity is a kind of exploration, perhaps, requiring trial and error, and therefore time invested. Art profs want to see portfolios, sketchbooks, incomplete works, anything that shows that the student is engaged. And the judgment of how much work one has done can be fairly objective; it's something you can talk to the student about and reach agreement on. Of course, the quality of the engagement counts as well, but to some extent I think that comes out also, if one can review all the cars in the whole train of thought. This is certainly true of teaching math at the upper levels. It's a thrill when a student comes to you with "I thought of this problem and tried to solve it. Will you look at it?" Whereupon you're presented with scruffy bits of paper (mathematicians will write on anything) with formulas all over them. Most of them are wrong--false starts. It's like what one of my art colleagues described looking at sketchbooks is like: it's raw and unprocessed, and more powerful than a finished work.

So it may be that there is a natural division between objective and subjective assessments and grades. The former are relatively easy. But maybe for more complex outcomes we need an approach more like that of art: look at not just a finished product on a test or paper, but demand to see the corpus of work, mistakes and all, that led to it. Technology can obviously help with this because information is cheap to store "forever." Portfolio systems as they are generally currently conceived are not really the right tool for this--what you'd want is a virtual space for storing documents and imposing a bit of structure on them. Perhaps a mind-map hyperlinked to documents and meta-data tags on the whole thing, so it can be sorted and presented by different facets. Add the ability for an instructor to freely annotate these nodes and artifacts with hidable notes, and it starts to sound attractive. At any rate, this is not the kind of problem that another scoopful of rubrics can solve.