Critical Thinking Buzz

Yesterday I promised to revisit the different points of view about critical thinking evidenced at the Assessment Institute. This is a big topic, and I've babbled about it before (here). It's important because so many institutions plug it in as a learning objective. It's especially dear and near to the circulatory organs of liberal arts schools.

First, I have some bad news. Just look at the graph of search hits from google trends to unveil the sad tale.
The story is clear: searchers are becoming less interested in thinking and more interested in ignorance, with the latter spiking in about June 2009. In case you think student engagement might be riding to the rescue, the sad little red line at the bottom says otherwise. Interestingly, searchers seem to lose interest in either aspect of cognition around the holidays. After much research, I discovered the reason behind this, shown below.

Okay, maybe that's enough silliness :-)

Here's what I heard about the subject at the conference. Three things. First, as I mentioned yesterday, the opening plenary revealed a divide among the panelists when the topic came up as a part of a wider conversation. I was writing furiously and am not sure who to attribute this statement to:

There is no evidence that there are generalizable skills like critical thinking. You have to master one domain first.

This echoes my own thoughts and classroom experiences on the matter, which I've described previously. In a nutshell: developing significant ability to do deductive reasoning is a prerequisite to doing interesting inductive reasoning. If this is true, major programs are more likely to cultivate complex thinking skills than a broad curriculum like general education.

Other panelists disagreed, but there wasn't time to have a proper debate on the issue.

The second encounter was in a session about integrated assessment, where gen ed-type skills like communicating and thinking are threaded throughout the curriculum. Critical thinking was explicitly one of those. During Q&A I offered my own (heretical, I suppose) thoughts on the topic and was rather sternly admonished that the problem had been solved by psychologists and all I had to do was look in the literature. Heck, that may be true, and I've started looking. I'm a math guy after all, not a psychometrician. But if there was overwhelming theory and empirical evidence for a particular model, why is there still debate? Is it like Darwinian evolution, where some simply reject it because of dogma? I doubt it, but I'll try to find out. In the meantime, color me dubious. I asked the session speaker for some references yesterday by email.

I did find an accessible article by Tom Angelo, who was on the plenary panel, called "Beginning the Dialogue: Thoughts on Promoting Critical Thinking." It was published in 1995, and the opens by saying about critical thinking that "Despite years of debate, no single definition is widely accepted." This was actually confirmed in the session I mentioned, where it was taken for granted that critical thinking in an English course is different from in a Math course. This by itself isn't fatal to the idea; after all, writing is different too, but we can try to teach writing across the curriculum. But the subtleties are important.

For one thing, on a very basic level, I can watch Tatiana write a paragraph and say with confidence "this student did some writing." Evidence of thinking is different. I can look at that same paper and try to imagine what went on inside her head when she wrote it, but I can't really know that she was thinking at all. Maybe she put random words on the paper, or quoted something she had memorized. There's an empiricism gap. I can count words written. Quantifying (even with a binary yes/no) critical thinking is not so straightforward. To push that a bit further, imagine taking a piece of work from an English class and sticking it into a stack of math papers being graded. The math prof squints at it and wonders what the heck this is. Can the prof then pronounce whether or not critical thinking has taken place in the English class by inspection? Or is he/she only competent to judge in the domain of math? Reverse the situation. An English instructor sees a complex page of handwritten formulas and text, purporting to settle the Continuum Hypothesis once and for all. If you don't have technical expertise in an area, it's virtually impossible to judge what level of thinking has occurred. But maybe that's not what we mean. Here's a definition Dr. Angelo likes in the article, quoting Kurfiss:

[Critical thinking is] an investigation whose purpose is to explore a situation, phenomenon, question, or problem to arrive at a hypothesis or conclusion about it that integrates all available information and that can therefore be convincingly justified. In critical thinking, all assumptions are open to question, divergent views are aggressively sought, and the inquiry is not biased in favor of a particular outcome.

Allow me to point out a couple of things here. First, the creation of a hypothesis is (if true) the creation of new knowledge. And because we want it to conform to facts and hold up to new evidence that arrives, this is very similar to inductive reasoning. It sounds like the scientific method. But in order to do any kind of inductive reasoning, you have to have some knowledge of the deductive processes active in that domain. You can't write a math proof without knowing propositional logic; you can't solve a problem the rudder on your 747 unless you know how the thing works; you can't create complex financial leveraging instruments unless you understand the risks. Well, maybe that last one was a bad example.

One curious aspect of critical thinking assessment is that although the language surrounding it mentions all kinds of desirable habits of mind, higher ed kind of dodges the issue. Maybe there's some institution out there that really tackles teaching of self-assessment, open-mindedness, thoroughness, focus, and so forth. I'd love to know. Outcomes like what a student wrote on a piece of paper is far downstream from the actual events that led to its creation. There are some surveys that try to assess habits of mind--the CIRP for one. Who's trying to teach them that outside of orientation class? There is therefore a curious disconnect between our actual desired outcomes and what we teach.

I think it's healthy to ask another level of why. Why do we want students to think critically? We get stuck in our own curricular bubbles, perhaps. Let's step outside for a moment. What is gained if critical thinking--however you define it--is employed?

Is it because we want active citizens? Or we want good problem-solvers? Or we want entrepreneurs? Whatever the answer is, it's likely to be more easily pinned down than the amorphous one of critical thinking. We can actually look at evidence of citizenship or problem-solving. We can create programs and curricula to address them. And don't forget the habits of mind thing--for my money, that's an untapped vein of gold, and also amenable to assessment. At the right point in the report cycle, the assessment director can still aggregate the heck out of all the types of "critical thinking" on record and serve up a glop of statistical goo to the admins. Just put the graphs in color--that means more than any data you put on there. And be sure you put the right logo on the thing.

The third encounter with critical thinking I had was indirect; I heard about it through a conversation. What I was told was that on a Tuesday session U. Phoenix presented served up "Assessment Methods: Creating a Critical Thinking Scoring Instrument as a Tool for Programmatic Assessment." I can't find the slides online, but I'll try to get them, and ask the presenters what happened. Apparently there was essentially heckling of the presenter(s) about the definition of critical thinking, and my companion's assumption was that this was related to the fact that it was U. Phoenix presenting and not, say, Alverno College. If so, this is a sad irony. If a group of professionals gather to talk about critical thinking and don't actually demonstrate the ability to do it, where are we? I will suggest to the organizers that they replace iced tea refreshments with Scotch next time.

Ignorance => Meta-Ignorance

In the last article here, I speculated about "unknown knowns," those bits of institutional knowledge that may be locked away by silos and rigid processes. I suggested that it might be in the institution's best interests to shake those out. It's a natural effect of a new administration taking over, or probably should be. Someone passed along the following advice about new administrations: keep the best one third of the current leadership, bring in one third new from the outside, and promote one third from within. It seems to me that this combinatorical shuffle would have the effect of breaking up old processes and modes of thought and allowing a temporary meritocracy of ideas to prevail. If only we could do that with the tax code!

It all surely comes down to the continued development of professional expertise of everyone on the job, I think. Encouraging subordinates to challenge our ideas may slow things down a bit occasionally, but in my experience is a good way to improve decisions. Isn't that what academia is all about anyway? You can't create new knowledge without challenging an existing mode of thought or 'best practice.' (The label 'best practice' makes me grit my teeth--surely any practice can be improved, no? It sounds like an admission of failure. 'Accepted practice' is more honest.)

Ignorance is meta is the conclusion of an article in the New York Times' science section from January 18, 2000 called "Among the Inept, Researchers Discover, Ignorance Is Bliss." The article suggests that there is a double-whammy to being uninformed. The ignorant don't know, and they don't know they don't know. That is, they are confident in their knowledge, even when they have little. Author Erica Goode explains:

One reason that the ignorant also tend to be the blissfully self-assured, the researchers believe, is that the skills required for competence often are the same skills necessary to recognize competence.

Cornell Psychology professors Dunning and Kruger, who researched this idea, make some interesting points, as quoted in the article:

• Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it.
• This deficiency in "self-monitoring skills," the researchers said, helps explain the tendency of the humor-impaired to persist in telling jokes that are not funny.
• Some college students, Dr. Dunning said, evince a similar blindness: after doing badly on a test, they spend hours in his office, explaining why the answers he suggests for the test questions are wrong.

If you have followed this blog on the topic of noncognitive assessment, you may recall that realistic self-appraisal is one of the predictors of success. On the other hand, the most able subjects in the study conducted by the researchers were the most likely to underestimate their own abilities.

The researchers attributed this to the fact that, in the absence of information about how others were doing, highly competent subjects assumed that others were performing as well as they were -- a phenomenon psychologists term the "false consensus effect."

There is some hope: Kruger and Dunning were able to 'train in' more realistic self-appraisal skills for those lacking them. The problem, they suggest, is lack of feedback. If you're doing a lousy job and no one tells you, how will you learn otherwise? A certain amount of humility is a good thing.

Of course, there has to be a balance. Paralysis through analysis is no good either. Being too timid to act on a new idea because there is no way to find out if it's good or bad prevents real leadership. After all, if all decisions are obvious, why are they paying you that fat administrative salary? Unfortunately, the Total Quality Management model that accreditors are fond of these days assumes that with enough information, good decisions can be made. That isn't always the case--just look at the stock market. A lot of very smart people with a lot of very good information get it wrong about half the time.

Therein lies the key to good leadership: entertaining new ideas on the one hand, but in spite of little information to go on, intuiting which of them are disastrous. I think this is a very rare trait. As Niccolo Machiavelli wrote in The Prince:

There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success than to take the lead in the introduction of a new order of things.

Readers of this column will not be too surprised that I have an anecdote to supply on this general topic. It was first published in a small college literary magazine, probably a decade ago. I will not insult your intelligence by highlighting my own examples of meta-ignorance in the story. You'll find them easily enough.

Geniosity

I’m a genius. Well, perhaps I should modify that statement just a tad. I was a genius. In fact, on two separate occasions during my life, I have had pure Eureka! moments that lifted me from the mundane to the ethereal. The descent was just as sudden, but at least I got a glimpse of what it must be like to be a real genius—you know, the kind that wakes up and goes to bed still in the bug-eyed goddamn I’m smart state. I suppose some people must find it addictive, having instant blinding flashes of that leave them gasping. I wouldn’t want it to happen all the time, though, or I might drive into a tree just as I’d solved the global deforestation problem, for example. All in all, I found it to be quite pleasant, although I noticed right away about how other people aren’t very interested in moments of clarity, unless it’s their own, in which case it’s hard to get them to shut up afterwards. So I got a bumper sticker that reads My dog had its day that I put right beside the Towers will be violatedand Don’t void where prohibited stickers. It’s a little obscure, but I figure that’s okay because obscurity is hard to tell from profundity sometimes.

It happened while I was doing dishes. The sink in my kitchen is divided into two stainless steel basins, with a faucet arm that can be swiveled to either side. Both sides drain down the same pipe. The problem is that every time you turn on the garbage disposer, which is attached to the right side, it backwashes filthy water up into the left basin. Since that’s usually where I place the dishes to dry, it’s a less than perfect situation. Before my instant of genius, I resorted to turning on the disposer in short bursts so as not to give it time to spew much water back up the other side. I had done this for years. But last week, I had finished stacking the last plate into the rack in the left basin, and was contemplating the pool of foaming dirty water in the right basin waiting to be drained when I had my geniosity (one of the perks of geniushood, even if only a part-timer, is the permission to create new words). I realized that if I ran some clean water into the left basin before turning on the garbage disposer, at the very worst only clean water would come back up! It worked beautifully, and I have switched entirely to my new method of draining the sink.

I was beginning to wonder if I’d lost the touch, because my only previous geniosity had occurred when I was in kindergarten, some thirty years before. There was the possibility that that earlier one had been a fluke, but now I’m convinced that if I wait another thirty years something equally profound will occur to me. I’m thinking of starting a newsletter. Anyway, back to kindergarten: it was one of those special days when something extraordinary happens. In this case, we had a magician coming to perform for us in the auditorium. I was hoping it would be the good kind—magicians that do magic tricks, instead of the bad kind—magicians that just play music. It was some time before I realized that musician is a whole different word. We were led into the auditorium in single file, and row-by-row filled up the folding chairs set up on the floor. They started with the back row, and I ended up in the second row, a prime spot for watching the tricks, if they were to materialize. As I planted myself into the child-sized folding chair, I noticed the kids who were being led into the row in front of me. The child about to sit directly before me was Bobbie. This was before last names were invented. Bobbie was a troubled child. He had announced one day on the playground that his real name was Robert, for which the rest of us laughed him to scorn. Really! We might have only been five, but we weren’t stupid enough to believe that you’d call a thing something other than what it was. A Bobbie was a Bobbie, and a Robert was something quite different.

As Bobbie prepared to sit, I had my geniosity. If I were to pull his chair back, I thought, he would miss it and end up on the ground! No sooner had inspiration struck than did I put it into action, and to my amazement it worked! Bobbie plopped right on to the floor, and then looked around with the most bewildered expression, which could be interpreted as how did I miss a twelve-inch wide chair with a six-inch wide butt? His universe had changed forever, as had mine. He had discovered The Unexplained, and myself a profound moral question:

Are some geniosities best left unimplemented?

Sadly, this question hasn’t gotten the attention it deserves, despite my well-received article (cf. “Gravitational effects of translation while sitting,” Kids Today 1969, v102, pp 87-89) which raised the issue. It is even more important today that it was then. Do you think those guys at Los Alamos, mucking around in the desert, really thought they could build an atomic bomb? Ironically, the typical defense used when a geniosity is misapplied is the stupidity claim. I’m ashamed to say that’s exactly what I used to explain Bobbie’s unexpected contact with the floor. The resulting Q&A with my teacher Miss Birdbalm is instructive. My comments are in brackets.

Q: Did you pull Robert’s chair out from under him? [Direct question, a tough nut.]

A: That’s not Robert it’s Bobbie! [First attempt—misdirection, obfuscation]

Q: Did you? [Miss Birdbalm was not easily distracted.]

A: Yes, but I didn’t know that Bobbie would sit on the ground. [The stupidity defense.]

Q: Why did you pull the chair back? [Her first mistake, questioning my intentions.]

A: I thought it would help. [Who can argue with good intentions?]

Q: How would pulling Bobbie’s chair back help him? [She’s a gonner now.]

A: I noticed that he walks with a limp, so I calculated the moment of inertia about his ankles and concluded that his head would be whiplashed against the back of the chair upon sitdown. Unfortunately I overcompensated and pulled the chair out too far. Believe me, it would have been worse if I’d done nothing at all. I’ve got all the bugs worked out now... [You get the idea.]

The problem with geniosities is that they are too precious not to be implemented, so that Whoa, I could do THIS!, is inevitably followed in short order by Whoa, I did THAT! Maybe this is the march of progress, but I’d like to propose a moratorium on geniosities until we get this sorted out. Except for mine, of course. I only have helpful ideas now.