Recursive Critical Thinking

I was going to entitle this piece "critical thinking squared" as a cute way to imply critical thinking about critical thinking, but the imprecision bothered me. Squared means multiplication by self, and that's not the same as applying a process to itself. What multiplication means in this context isn't precise either, but you can possibly make a sense of it by considering a combinatorial factorization into dimensions like critical thinking = (analysis, creativity, communication). If we abbreviate critical thinking = CT, then CT² might look like a matrix:

	Analysis	Creativity	Communication
Analysis	Analysis	Creativity * Analysis	Communication * Analysis
Creativity	Analysis * Creativity	Creativity	Communication * Creativity
Communication	Analysis * Communication	Creativity * Communication	Communication

This assumes that each dimension is idempotent (meaning S*S = S), and that "*" is some way of combining the two dimensions. You still have to figure out what the Creativity * Analysis combination means, but at least you have a way to produce detail from the squaring operation. But this is all rather silly, and the reason I don't like the "CT squared" idea.

Here's a better way to think about it. If you talk about talking, that isn't (talking)², but rather talking(talking), expressed here as a function that takes itself as an input.

This is even practical. For example, you could write a function get_loc(...) in the C programming language to take a function and return its address in memory. Then you could ask for get_loc(get_loc) to retrieve its own location. This sort of thing is called recursion, and it's a big deal in computer science. In common parlance, we might stick the prefix meta- in front of the concept to show that it's recursive, as in metacognition, which in the right context we justifiably call a noncognitive trait: the reflective practice of thinking about one's own thinking process. More on the relationship between CT and noncogs later. First, let's take a closer look at the role of recursion in problem solving.

In a few stolen moments this morning I was sipping an iced latte, enjoying a cool breeze, and trying to make some progress on a research project regarding survival in a certain abstract sense. You can read the actual paper here, but in a nutshell you can imagine an environment that poses survival challenges to organisms (all abstracted into computer language, which you can learn more about and download a simulator here). There are two different questions regarding the complexity of a given environment:

1. What's the simplest thing that can survive the given conditions?
2. What's the simplest recursive process that can find the solution to #1?

Here, recursion means that some process can be tried over and over again, feeding the output of the last iteration into the input of the next. Like natural selection, for example, acting recursively on the gene pool to blindly hone the fitness of the survivors.

Let me give a more down-to-earth example, as a simple "critical thinking" problem. Suppose Tatiana works all day in retail, and part of her job is to calculate sales reductions for coupons, sale prices, and so on. In addition she has to add sales tax to total amounts. For reasons known only to management, they skimped on her point of sale (cash register) and these functions are not included. So all day long she has to do stuff like:

1. Find actual cost of an item by reducing for sale price, coupon, etc.
2. Sum adjusted prices
3. Calculate tax
4. Add to get total

This is pretty tedious and prone to error, so there's an advantage to having the best possible way of doing this. In the context of my framing questions, we should ask:

1. What's the best way of doing her job?
2. How do we find it?
   

In practice, we have to address the second before the first. The second we might call a critical thinking exercise, requiring analytical and creative thought.

Tatiana need not be reflective. She probably already has a solution, and may not care that it's not optimal. I see this all the time in real stores. A clerk wants to reduce an item by 15%, say. Most often they multiply the price by 15% (.15) on a calculator, write down this number, and then subtract from the original price. Sometimes I tell them a quicker way to do it: just multiply the original price by .85 and you're done.

In a complex environment, you're never really finished with the "how do we find a better solution?" step. The question itself is recursive: "how do we find better ways of finding better solutions?" Mathematics is full of this sort of thing. You can follow the chain of meta-thought all the way up to something called category theory, where logic itself can be generalized (logic about logic).

So without really thinking about the definition, the idea of "critical thinking" can get you rapidly into the deep part of the pool. For me, it's very important to keep straight the difference between knowing a good solution to a problem and finding a good solution to a problem. I don't think this is as appreciated as it should be. The first is analytical/deductive and the second is creative/inductive, and they require very different preparations.

Think of an old-timey telephone switchboard operator, plugging and unplugging wires all day.
(photo courtesy of Wikipedia). There's a vast difference between knowing how to operate the switchboard and knowing how to design one or improve existing designs. The link between these two questions, as with the ones above is the "why" operator. Here's a possible chain of why-iterations for thinking about telephones:

1. Q: Why are you moving those wires and plugs around?
   A: I'm operating a switchboard according to the procedures I've been trained in
   
2. Q: Why is there a switchboard?
   A: To facilitate telephone calls.
   
3. Q: Why are telephone calls useful?
   A: So people can communicate across distances.
   
4. Q: Why do people need to communicate across distances?
   A: So they can lead better lives.
5. Q: Why do people need to live better lives?
   

Each one of these increasingly general domains has its own problems and solutions. Solving the general ones can make the specific ones go away. If we keep asking why (see this related article), we end up with very general questions like "what problem does my existence solve?" and "why does anything exist?" I've tried to portray this recursion graphically below.
I'm assuming that the creation of knowledge is scientific (what art does is something different from what I mean here). I've quoted Bertrand Russell before on this point (here):

All definite knowledge--so I should contend--belongs to science; all dogma as to what surpasses definite knowledge belongs to theology. But between theology and science there is a No Man's Land, exposed to attack from both sides; this No Man's Land is philosophy.

Philosophy, theology, and other avenues of inquiry that remain immune to the scientific method are lumped together at the bottom of my graph. Wouldn't it be nice if we showed our students of critical thinking how this works? The unveiling of the breadth of meta-thought ought to be a stunning moment of realization for an undergraduate. Consider the following question and meta-question:

Why is the sky blue? (proposed answer here)
Why ask why?

From a scientific question we arguably leap directly over all of science to a philosophical one. Not only that, to my eyes it seems like a fixed point under meta-recursion. That is:

Why ask "why ask why?"? is the same as Why ask why?

Which would make the question the most profound one possible, I suppose. This could be a great starting point for a course on critical thinking. Note that I kind of cheated in my one-step leap to "why ask why?" Figuring out how is your meta-cognition homework. :-)

How does all this fit with existing literature on critical thinking? A colleague recently pointed me to the 1988 publication "The Delphi Report" on critical thinking, which arguably kicked off recent interest in the teaching and assessment of said skill. In the executive summary, which is linked to the title, a consensus statement reads:

We understand critical thinking to be purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based.

This is quite different from the line I've taken above, isn't it? Maybe it's not even useful to use the term "critical thinking" for both. In the above definition, which is the kind usually used, it's described as an activity with a particular type of result. The activity itself is not described here other than purposeful, self-regulatory judgment. These are noncognitive descriptors, please note. In fact, the definition elaborates on this point with a vivid description of the thinker:

The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fairminded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit.

As far as I can tell, in practice most programs in critical thinking don't actually pay much attention to the "affective" or noncogitives listed. But that's not too unexpected--academics in general seems to be allergic to modeling and teaching personal attributes. I find it increasingly odd that this is so.

Into the meat of the executive summary we do find particular cognitive skills. You'll see these or similar ones in rubrics and learning taxidermy.

1. interpretation
2. analysis
3. evaluation
4. inference
5. explanation
6. self-regulation

I don't see how self-regulation is cognitive, but maybe it is in a metacognition sort of way (self-reflection). One of the findings is that evaluating one's own thinking is a way to improve it.

Although they don't get around to saying it this way, the authors note the importance of analytical skills:

Although the identification and analysis of CT skills transcend, in significant ways, specific subjects or disciplines, learning and applying these skills in many contexts requires domain-specific knowledge.

Knowing how to solve an urgent problem while sailing is different from solving one while flying a plane.

This debate is important. Lots of institutions put "critical thinking" on their to-do list. Good definitions should lead to good implementations and good assessments.

Although I appreciate the value of the work that's been done in traditional meta-critical thinking, I don't much like the result--those lists of vague terms like interpretation and evaluation. I know they can be used successfully, and they can probably produce a good curriculum and assessment. But to me they're just a disjoint collection of loosely-defined techniques that a committee came up with. There's no underlying structure or theory. No way to make sense of it all by asking the meta-question: why is critical thinking the way it's described in "The Delphi Report?" You can only answer that the experts agreed that this is what it should be. In Russell's description, this makes it dogmatic. And if critical thinking has any value at all, it's to question dogma, no? That makes it ironic, but we can't judge too harshly on this account. Even Karl Popper freely admitted that his system could not be proven to be self-consistent (i.e. prove that nothing is every really proven). You have to start somewhere.

It may just be my bias coming from a computer science/math background, where a more natural schema is the study of algorithms and complexity, but I'd like more than the opinion of a panel of experts. I want to keep asking why until there is a self-consistent answer, if possible. In the meantime, here's my recipe for teaching CT:

1. Analytical thinking in various domains
2. Creative thinking in domains where analytical thinking is well-developed by the student
   
3. Training in recursive thought [ including CT(CT) ]
   
4. Communications skills (otherwise, what's the point?)
   
5. Noncognitives like intellectual honesty and open-mindedness and willing use of the above skills.
   

I know the first two can be effectively assessed; I've never tried to do the third because it just occurred to me today. The last one ought to be on everyone's to-do list.

What's Wrong with Standardizing Minds

I read Peter Sacks' Standardized Minds a few years ago when I was helping put together our general education assessment process. This led me to look for others attempting to counter the standardization of education, and I found Alfie Kohn [website]. I corresponded with him briefly about some of the issues, and sent him our subjective assessment techniques. On the front page of Reddit today (it's embarassing how much of my news comes from this geeky aggregator) is a parody by Alfie of the standardized testing process. From the fake ad:

With ADS, you get minute-by-minute teaching guides, thousands of practice tests, and other materials for turning your school into a 21st-century test-preparation factory.

The whole thing is pretty cringe-worthy, because like any good parody it's not far from truth. For example, the SAT critical reading section gives the test-taker a chance to show how well she understood what she read. From the College Board's SAT test preparation web page:

There are two types of multiple-choice questions in the critical reading section:

• Sentence completion questions test your vocabulary and your understanding of sentence structure. (19 questions)
• Passage-based reading questions test your comprehension of what is stated in or implied by the passage, not your prior knowledge of the topic. (48 questions)

Here, 'critical reading' has been reduced to simple understanding of what you've read. No wonder, since actual critical thinking is such a mess to assess.

So it's easy to take shots at the current methodology of mass production testing, which I refer to sometimes as neo-phrenology, but is it so easy to put your finger on the exact problem? Obviously, we want to know whether or not the students are learning, and so we test them. If the tests and test conditions are all the same, then it's fairer than testing willy-nilly. Two short steps, and we have arrived at industrial testing. So where did we go wrong?

The problem is a nasty little enthymeme.* Instruction and testing, as I experienced it in school, and as my daughter is now, focus primarily on analytic/deductive processes. That is, low-complexity thinking. Let me explain.

Suppose you take a quiz on US state capitals, and you know 49 of them. How hard it is for you to figure out the 50th from your knowledge of the 49? It's clearly impossible--there is no information in the first 49 that contributes to the knowledge of the 50th. In informational terms, the names are random. This just means that you have to know it because it's not figureable out. In other words, you can't get there by generalizing.

Much of education is of this sort. What date was the Pearl Harbor attack? You either know or you don't. It's not like you can go scribble in the corner for a while and come up with the answer through sheer mental effort.

Some problems are solvable from given information, of course. This is largely in the sciences, where some raw materials are given, and the solution to a problem is latent in those materials. Chemical reactions, physics modeling, math word problems, and the like all depend on more advanced forms of deductive reasoning (or analytical thinking if you like). This can become quite sophisticated, such as solving systems of linear differential equations by finding eigenvalues of matrices, and so forth. But this is still not generalizing, it's just applying more complicated methods of analysis--the rules are more dense.

The hidden assumption inherent to standardized testing is that if students are good at analytical/deductive thinking, they will be good at creative/inductive thinking that leads to insightful generalization. That is, that students who test well will be good at generalizing problem solving. This skill is much harder than analytical reasoning, because there is only trial and error, seasoned by experience, as a guide. And yet the ability to solve problems creatively, generalizing from what one already knows, is arguably the most prized kind of thinking; this is where new knowledge comes from.

In science class, students sometimes taught the scientific method by looking at information and forming hypotheses. Literature classes are hard because of the fuzziness inherent in generalized, creative thinking. Same for geometry, where analytical methods are taught as a conduit to creative proof construction. These are valuable experiences because they teach new modes of thought, essentially finding patterns and exploiting them.

It's hard to use a fill-in-the-bubble test to look for generalized thinking because the answers are right in front of you. The whole point of a creative endeavor is that the answer isn't given, and may not even exist! Even free-form elements like writing a short essay are subjected to analytical measures when scoring in a standardized context. How do you teach a computer to measure creativity, after all?

By placing so much importance on standardization, we emphasize rote problem solving and step-by-step methods--the hallmarks of deductive reasoning. This is not unimportant--one does need to know ones times tables--but it's setting the bar too low. Conveniently, it's far easier to assess and teach analytical skills. Unfortunately, it misses the larger point. We need thinkers who can do the following:

1. Look at what's given and recognize connections because of analytical training
2. Form ideas about patterns that may exist and test these guesses against what is known
3. Generate paths of inquiry to find new information that solidifies or overturns existing knowledge
   

Standardized testing as it is commonly done only addresses the first point. If our students really learn to think, it will not be because of test-preparation.

---

*I think I may have gotten that line from Giles Goat Boy (another parody of higher ed) by John Barth. Highly recommended.

Struggling with the Idea of Critical Thinking

I came across an excellent article (pdf) in Psychological Science by Daniel T. Willingham entitled "Critical Thinking" and subtitled Why Is It So Hard to Teach? The author first presents a case for the teaching of critical thinking as an integral part of the nation's educational product. This skill is defined "in layperson's terms" as

[S]eeing both sides of an issue, being open to new evidence that disconfirms your ideas, reasoning dispassionately, demanding that claims be backed by evidence, deducing and inferring conclusions from available facts, solving problems, and so forth.

It is also noted that there are particular kinds of critical thinking linked to academic pursuits, such as history or science. Here's an example of literary criticism (a review, in fact, so it's meta-criticism), chosen at random from Critical Inquiry (Spring 2003, pg 463):

[The author] demonstrates a thorough historical awareness of the obscene cruelty of the Stalinist and Fascist regimes and, in fact, suggests that the profoundly stupid cruelty of the totalitarian regimes is more theoretically pertinent than theory itself in the dark illumination it casts on the institutions of the West, where the term totalitarian is often used, in a way that to [the author] suggests gross complacency, emplifies the illusion of freedom and agency in contemporary democracy, illusions unavailable in non-democratic societies, where the truth is confronted more directly in the from of a short circuit between authority and brutality.

There are many other examples from a variety of disciplines that we could choose, and they will look different from one another. What is this "critical thinking" stuff that unites them? By the way, I'm not sure "emplifies" is really a word, but that's part of the fun.

Defining the process of critical thinking takes up a good bit of Willingham's piece. He presents the idea that there is a difference between superficial and deep structure, and that humans naturally cling to surface structure. Through training we can see through the surface structure and generalize problem solving. An interesting example is given.

A treasure hunter is going to explore a cave up on a hill near a beach. He suspected there might be many paths inside the cave so he was afraid he might get lost. Obviously, he did not have a map of the cave; all he had with him were some common items such as a flashlight and a bag. What could he do to make sure he did not get lost trying to get back out of the cave later?

The "correct" solution is to put some sand in the bag to use as markers within the cave, to be used to retrace the route back. What's interesting is that when this problem was posed to American and Chinese college students, 75% of the former latched onto the solution, but only 25% of the latter. The hypothesis is that this problem has the same deep structure as that of the story of Hansel and Gretel, which is common to American culture but not Chinese. That is, because of prior exposure, American students could see through the distracting surface structure (beach, cave, etc.) to see the underlying problem (the need to leave a trail behind oneself).

Even if one has been trained in deep structure, like mathematical methods of solving problems that may all look the same, but are really the same logically, one must also be trained to use this knowledge, argues the author--we have to know when to look for deep structure. The reverse is true also--even if one knows to look for deep structure, one must know how to do it. Knowledge of content and methods of inquiry are both prerequisites. But are they sufficient?

Critical thinking is not a skill at all is the hypothesis advanced later in the article, after a summary and discussion of the difficulties and general failure to prove that teaching critical thinking per se has been successful.

Critical thinking does not have certain characteristics normally associated
with skills—in particular, being able to use that skill at any time. If I told you that I learned to read music, for example, you would expect, correctly, that I could use my new skill (i.e., read music) whenever I wanted. But critical thinking is very different. [...] [P]eople can engage in some types of critical thinking without training, but even with extensive training, they will sometimes fail to think critically.

Here I will take my departure from the article and offer my own thoughts. I think the idea presented in the quote above is absolutely on the money. To illuminate, consider the following example. An employee works on a project for his supervisor, and presents the results. The latter reviews the work carefully and pronounces "you did a good job." This is obviously a desired outcome for the employee. Is it something we can teach our students: to do a "good job"? No. The reason is that this is a description of the outcome, and is not directly related to the process of attaining it. Note also that agreeing that the description is accurate does not give us license to assume that we can measure the accomplishment with some metric. This assumption is, unfortunately, blithely made very often in academic assessment circles.

We can understand this problem better by looking at its deep structure. Some abstract problems have very difficult solutions, which are nevertheless easily confirmed once found. Suppose I set up a row of cups numbered 1-1000 and secretely put a pea under one of them. Your job is to figure out where the pea is by turning over cups and looking. Generally speaking, you'll have to turn over 500 cups on average to find it. There are no shortcuts. But once it's found, verification is obvious. You'll know instantly that you did a "good job" if you find the legume on the second try. In computer science this phenomenon is well known, and even famous. An outstanding example is the P=NP problem.

Hard to construct, easy to verify describes many of the outcomes we observe in our professional lives. That's why experience and learned judgment are invaluable in most, if not all, professions. If we accept that critical thinking should not be thought of as a skill that can be taught but rather a judgment about efforts made, then how can we proceed? What can be taught?

I've argued before that critical thinking is a fuzzy concept, so maybe my reading of this article is just confirmation bias on my part, but that won't stop me from presenting my argument anew, framed in the current context.

The idea that knowledge and skills are both required to develop cognitive skills was a point mentioned earlier. But we can be more specific without becoming pedantic or falling into a mereological fallacy. Knowledge and algorithmic skills can be lumped together into a basket called analytical thinking, or if you prefer, deductive reasoning. Knowing facts is the simplest kind of deductive reasoning, but it also includes analyzing chemical processes according to given rules, solving all kinds of math problems, checking grammar, and so on. We spend a lot of time in education trying to convey this kind of knowledge.

Underappreciated, I believe is the complementary cognitive skill of inductive reasoning. During a college curriculum, this type of thinking sneaks up on a student, and he or she may fail simply because of the lack of warning. I think instructors mostly aren't aware themselves when they cross the threshold from analytical/deductive to what I'll call creative/inductive.

Creativity and induction are both generalizing operations: the creation of new knowledge for the learner. When I was in grade school, working with decimals and fractions for the first time, I remember creating the following hypothesis after working enough problems: the reciprocal of an integer n near 10 is about .10 - (n - 10)/100. This looks complicated, but in practice, if I wanted to remember what 1/13 was, I'd take the '3' from 13, turn it into .03 and subtract from one tenth to get .07. The actual reciprocal is .077, so it's not a bad approximation. On the whole, this is a lousy way to get the reciprocals, though, because it only works in a narrow range. My brain is just wired to remember processes rather than facts, so it stuck.

The point of this numerical digression is to illustrate the creative/inductive thought process. It's messy, often inaccurate, and a lot of work to get right. Unlike the step-by-step process of teaching an analytical/deductive skill, there are false starts and inevitable setbacks. There's not even a guarantee that there is a solution. This takes a whole different outlook on the part of the problem-solver. Students who are used to the analytical process, and who suddenly encounter the trial-and-error necessities of a creative process are often stymied. But I think it can be taught, and the first step is to recognize the problem and address it.

Unlike the fuzzy label "critical thinking," which arguably is not a skill at all, but an outcome of skills and knowledge, both analytical/deductive and creative/inductive skills can be taught and assessed. We just have to do so intentionally. Critical thinking as a skill to be taught and assessed is a meme that needs to go away... The sooner the better.

AAC&U's LEAP Initiative and Thinking Skills

If you haven't seen it yet, it's worth looking at the AAC&U's long-awaited LEAP recommendations for the liberal arts. (The link is a pdf to the executive summary). There's a lot there, but will just comment on the thinking skills part, shown below.

Later on in the summary, they cite some statistics about what employers wish for in graduates:

Notice anything about the two lists? The thinking skills are permuted somewhat, but they're still there. I'm interested particularly in creative and analytical thinking. In the guidelines these are combined. In the survey data, they are separated. They really should be separated in the guidelines too, because they're vastly different modes of thought. Both of the lists could use some editing. Ideally, the first two of the AAC&U's recommended list would be:

• Analytical thinking
• Creative thinking

I'm not sure what 'inquiry' is supposed to mean or how it's supposed to be taught, so let's toss that one. Much worse is the insidious 'critical thinking' skill that many institutions have in their set of goals. Granted, it sounds good--who doesn't want their graduates to be able to think critically. But what, exactly does it mean? It's far to fuzzy to be useful. I hope to convince you of this by contrasting it to analytical and creative thinking as a working dichotomy that can cover all our cognitive bases. Here are my working definitions for the purposes of curriculum development and assessment:

Analytical Thinking includes knowing facts and how they relate to each other. It includes definitions and languages and rules about how they work. For example, one can imagine a field of knowledge as a semantic field over which manipulations are performed explicitly. To the extent this is true, it is an exercise in analytical thinking. Analytical thinking is algorithmic: information retrieval and manipulation. It derives from deductive reasoning: consequences follow from given rules. In math, finding the derivative of a function is an exercise in analytical thinking. Identifying a piece of music as classical is analytical. Deriving the name of an organic molecule is analytical. Determining what a computer program does is analytical. Note that the rules can become very complex, and so there's no limit to the difficulty of analytical reasoning.

Creative Thinking is inductive or random. It looks for patterns and formulates them. It compresses complexities into simplicities, or does the opposite. It does this in the context of a body of analytical knowledge. Solving a known problem using known methods is not creative--it's analytical. Finding a new way to solve the same problem is creative.

In summary, analytical thinking is knowing facts and applying rules. Creative thinking is creating new facts and new rules. Without a background in some body of analytical thought, it's not possible to be productively creative. This models wonderfully well the way we teach and construct curricula.

Consider. First we seek to teach students the language of our discipline, and facts about the objects they encounter. We teach them theories about these, and show them how to apply theories. This is the analytical stage of learning. Some students may do very well with this. If they have a good memory and are good at following rules, they'll be good analytical thinkers.

Then, in many disciplines there's a shift. It's subtle, but devastating to some students, particularly if they haven't been warned, or if the instructors aren't aware. We begin to expect students to apply theories to new situations, or to create their own objects and theories. We're surprised when what we see initially looks random. Student try to mimic our process, but process takes them only so far--there's something else required: the magic of the human brain in generalizing, applying inductive reasoning, and the flash of insight or just sheer audacity of thought that distinguishes the best thinkers.

Some students have this naturally--this ability to insert randomness in a controlled way to create useful novelty. Others will flail around producing garbage. It's essential that they have some mastery of the analytical rules and knowledge of the discipline, or they can't edit themselves. They don't know right from wrong, good from bad if they don't have the analytical skills.

We as instructors can prepare students for this, if we are ourselves aware of this divide. For me, it came in a class called Introduction to Analysis, where I was expected to come up with math proofs on my own for the first time. My instructor was intuitive enough to know this was a hard class, and helped us enjoy the process, difficult as it was for most. But she didn't really understand, I think, why it was difficult. It was the transition from analytical to creative thought. I know this now, and preach it to my students. I even mark problems in the homework as creative or analytical. It's an extremely useful idea for organizing courses and curricula, in my experience. We assess it too, in a gentle way that doesn't require lots of tedious bureaucracy.

So there you have it. Critical thinking, in my opinion, is some confusion of analytical and creative processes, and is not a useful dimension for a general classification of cognition. It might be a great thing to focus on in an art or performance class, as a specific skill to be developed, but not as a first tier red-letter (i.e. rubric) goal.