Sunday, September 13, 2009

Recursive Critical Thinking

I was going to entitle this piece "critical thinking squared" as a cute way to imply critical thinking about critical thinking, but the imprecision bothered me. Squared means multiplication by self, and that's not the same as applying a process to itself. What multiplication means in this context isn't precise either, but you can possibly make a sense of it by considering a combinatorial factorization into dimensions like critical thinking = (analysis, creativity, communication). If we abbreviate critical thinking = CT, then CT2 might look like a matrix:

AnalysisAnalysisCreativity * AnalysisCommunication * Analysis
CreativityAnalysis * CreativityCreativity Communication * Creativity
CommunicationAnalysis * CommunicationCreativity * CommunicationCommunication

This assumes that each dimension is idempotent (meaning S*S = S), and that "*" is some way of combining the two dimensions. You still have to figure out what the Creativity * Analysis combination means, but at least you have a way to produce detail from the squaring operation. But this is all rather silly, and the reason I don't like the "CT squared" idea.

Here's a better way to think about it. If you talk about talking, that isn't (talking)2, but rather talking(talking), expressed here as a function that takes itself as an input.

This is even practical. For example, you could write a function get_loc(...) in the C programming language to take a function and return its address in memory. Then you could ask for get_loc(get_loc) to retrieve its own location. This sort of thing is called recursion, and it's a big deal in computer science. In common parlance, we might stick the prefix meta- in front of the concept to show that it's recursive, as in metacognition, which in the right context we justifiably call a noncognitive trait: the reflective practice of thinking about one's own thinking process. More on the relationship between CT and noncogs later. First, let's take a closer look at the role of recursion in problem solving.

In a few stolen moments this morning I was sipping an iced latte, enjoying a cool breeze, and trying to make some progress on a research project regarding survival in a certain abstract sense. You can read the actual paper here, but in a nutshell you can imagine an environment that poses survival challenges to organisms (all abstracted into computer language, which you can learn more about and download a simulator here). There are two different questions regarding the complexity of a given environment:
  1. What's the simplest thing that can survive the given conditions?
  2. What's the simplest recursive process that can find the solution to #1?
Here, recursion means that some process can be tried over and over again, feeding the output of the last iteration into the input of the next. Like natural selection, for example, acting recursively on the gene pool to blindly hone the fitness of the survivors.

Let me give a more down-to-earth example, as a simple "critical thinking" problem. Suppose Tatiana works all day in retail, and part of her job is to calculate sales reductions for coupons, sale prices, and so on. In addition she has to add sales tax to total amounts. For reasons known only to management, they skimped on her point of sale (cash register) and these functions are not included. So all day long she has to do stuff like:
  1. Find actual cost of an item by reducing for sale price, coupon, etc.
  2. Sum adjusted prices
  3. Calculate tax
  4. Add to get total
This is pretty tedious and prone to error, so there's an advantage to having the best possible way of doing this. In the context of my framing questions, we should ask:
  1. What's the best way of doing her job?
  2. How do we find it?
In practice, we have to address the second before the first. The second we might call a critical thinking exercise, requiring analytical and creative thought.

Tatiana need not be reflective. She probably already has a solution, and may not care that it's not optimal. I see this all the time in real stores. A clerk wants to reduce an item by 15%, say. Most often they multiply the price by 15% (.15) on a calculator, write down this number, and then subtract from the original price. Sometimes I tell them a quicker way to do it: just multiply the original price by .85 and you're done.

In a complex environment, you're never really finished with the "how do we find a better solution?" step. The question itself is recursive: "how do we find better ways of finding better solutions?" Mathematics is full of this sort of thing. You can follow the chain of meta-thought all the way up to something called category theory, where logic itself can be generalized (logic about logic).

So without really thinking about the definition, the idea of "critical thinking" can get you rapidly into the deep part of the pool. For me, it's very important to keep straight the difference between knowing a good solution to a problem and finding a good solution to a problem. I don't think this is as appreciated as it should be. The first is analytical/deductive and the second is creative/inductive, and they require very different preparations.

Think of an old-timey telephone switchboard operator, plugging and unplugging wires all day.
(photo courtesy of Wikipedia). There's a vast difference between knowing how to operate the switchboard and knowing how to design one or improve existing designs. The link between these two questions, as with the ones above is the "why" operator. Here's a possible chain of why-iterations for thinking about telephones:
  1. Q: Why are you moving those wires and plugs around?
    A: I'm operating a switchboard according to the procedures I've been trained in
  2. Q: Why is there a switchboard?
    A: To facilitate telephone calls.
  3. Q: Why are telephone calls useful?
    A: So people can communicate across distances.
  4. Q: Why do people need to communicate across distances?
    A: So they can lead better lives.
  5. Q: Why do people need to live better lives?
Each one of these increasingly general domains has its own problems and solutions. Solving the general ones can make the specific ones go away. If we keep asking why (see this related article), we end up with very general questions like "what problem does my existence solve?" and "why does anything exist?" I've tried to portray this recursion graphically below.
I'm assuming that the creation of knowledge is scientific (what art does is something different from what I mean here). I've quoted Bertrand Russell before on this point (here):
All definite knowledge--so I should contend--belongs to science; all dogma as to what surpasses definite knowledge belongs to theology. But between theology and science there is a No Man's Land, exposed to attack from both sides; this No Man's Land is philosophy.
Philosophy, theology, and other avenues of inquiry that remain immune to the scientific method are lumped together at the bottom of my graph. Wouldn't it be nice if we showed our students of critical thinking how this works? The unveiling of the breadth of meta-thought ought to be a stunning moment of realization for an undergraduate. Consider the following question and meta-question:
Why is the sky blue? (proposed answer here)
Why ask why?
From a scientific question we arguably leap directly over all of science to a philosophical one. Not only that, to my eyes it seems like a fixed point under meta-recursion. That is:
Why ask "why ask why?"? is the same as Why ask why?
Which would make the question the most profound one possible, I suppose. This could be a great starting point for a course on critical thinking. Note that I kind of cheated in my one-step leap to "why ask why?" Figuring out how is your meta-cognition homework. :-)

How does all this fit with existing literature on critical thinking? A colleague recently pointed me to the 1988 publication "The Delphi Report" on critical thinking, which arguably kicked off recent interest in the teaching and assessment of said skill. In the executive summary, which is linked to the title, a consensus statement reads:
We understand critical thinking to be purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based.
This is quite different from the line I've taken above, isn't it? Maybe it's not even useful to use the term "critical thinking" for both. In the above definition, which is the kind usually used, it's described as an activity with a particular type of result. The activity itself is not described here other than purposeful, self-regulatory judgment. These are noncognitive descriptors, please note. In fact, the definition elaborates on this point with a vivid description of the thinker:
The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fairminded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit.
As far as I can tell, in practice most programs in critical thinking don't actually pay much attention to the "affective" or noncogitives listed. But that's not too unexpected--academics in general seems to be allergic to modeling and teaching personal attributes. I find it increasingly odd that this is so.

Into the meat of the executive summary we do find particular cognitive skills. You'll see these or similar ones in rubrics and learning taxidermy.
  1. interpretation
  2. analysis
  3. evaluation
  4. inference
  5. explanation
  6. self-regulation
I don't see how self-regulation is cognitive, but maybe it is in a metacognition sort of way (self-reflection). One of the findings is that evaluating one's own thinking is a way to improve it.

Although they don't get around to saying it this way, the authors note the importance of analytical skills:
Although the identification and analysis of CT skills transcend, in significant ways, specific subjects or disciplines, learning and applying these skills in many contexts requires domain-specific knowledge.
Knowing how to solve an urgent problem while sailing is different from solving one while flying a plane.

This debate is important. Lots of institutions put "critical thinking" on their to-do list. Good definitions should lead to good implementations and good assessments.

Although I appreciate the value of the work that's been done in traditional meta-critical thinking, I don't much like the result--those lists of vague terms like interpretation and evaluation. I know they can be used successfully, and they can probably produce a good curriculum and assessment. But to me they're just a disjoint collection of loosely-defined techniques that a committee came up with. There's no underlying structure or theory. No way to make sense of it all by asking the meta-question: why is critical thinking the way it's described in "The Delphi Report?" You can only answer that the experts agreed that this is what it should be. In Russell's description, this makes it dogmatic. And if critical thinking has any value at all, it's to question dogma, no? That makes it ironic, but we can't judge too harshly on this account. Even Karl Popper freely admitted that his system could not be proven to be self-consistent (i.e. prove that nothing is every really proven). You have to start somewhere.

It may just be my bias coming from a computer science/math background, where a more natural schema is the study of algorithms and complexity, but I'd like more than the opinion of a panel of experts. I want to keep asking why until there is a self-consistent answer, if possible. In the meantime, here's my recipe for teaching CT:
  1. Analytical thinking in various domains
  2. Creative thinking in domains where analytical thinking is well-developed by the student
  3. Training in recursive thought [ including CT(CT) ]
  4. Communications skills (otherwise, what's the point?)
  5. Noncognitives like intellectual honesty and open-mindedness and willing use of the above skills.
I know the first two can be effectively assessed; I've never tried to do the third because it just occurred to me today. The last one ought to be on everyone's to-do list.


  1. Anonymous10:50 AM

    I've been thinking about this for a day and I believe the real issue goes to two problems that researchers have bumped into in Quantum Physics. The first is the Uncertainty Principle -- you can know with precision where something is or how fast/where it is going but not both and the more precisely you know one the less precisely you know the other.

    As it applies here, you can either study the outcome of the process of learning to think Critically or the direction of the process -- not both.

    The second is the famous double-slit experiment with light and how, in one experiment, it behaves like a wave and in another it behaves like a particle. The end result is that the observer determines the results of the experiment and not the "objective" reality being observed.

    Most of the research I have looked at (which is admittedly a very small amount) concerning assessment in Higher Ed appears to not want to deal with the issue that the moment you begin to measure something when it comes to learning, you impact the outcome (Your post from today -- 15 September 2009 for those coming to the conversation later -- for examples.)

    My suspicion is that until those who would research in this area are willing to think through the implications of how those two revelations in physics might change the way they look at assessment of student learning outcomes, there won't be much progress in assessing. In the end, the assessment of a complex system must inherently be complex (as it must take into account multiple variables within the context of their system) if you wish to capture that complexity. If you just want to know where students are, it can be simple. If you want to know where they are going, it can be simple. If you want to know both, it has to be complex.

    Matt DeForrest, who is not with it enough to figure out how to post the comment as anything other than "Anonymous" today.

  2. Matt, even without invoking quantum strangeness, there's a lot of fuzziness around any formal learning objective. That's why I don't like the word "measurement" for this sort of thing.

  3. I follow the logic, as this piece really tries to keep the argument on a logical course. Here are my thoughts:
    "We understand critical thinking to be purposeful, self-regulatory judgment WHICH results in interpretation, analysis, evaluation, and inference". In critical thinking the word 'which' IS determinism. To figure out how or why you chose 'which' is critical (a good reason). The bifurcation that Matt points out doesn't have to go to the sub-atomic level, but he does have an interesting allegory there. When the investigator gets trapped in a recursive mindset figuring out the why and how he/she misses the point. The point is to use some intellectual rigor before concluding which to chose...thanks for post (and sorry for the late input)

  4. I think of the 'which' in the quote as being a qualifier (should be 'that'). Like I want a car THAT goes 200 mph. In that vein, it means a sort of purposeful, self-regulatory judgment that is also going to produce the indicated results. This admits that other possibilities exist--there are other kinds of purposeful self-regulatory judgment. Keeping in time while playing with a band, for example. I just noticed that everything on the list is introspective too--critical thinking doesn't result in a direct call to action; it's purely intellectual.

  5. I think I see where you are going here; there isn't a call to action per se, but judges judge, and analysts analyze, and evaluators evaluate so these intellectual exercises can lead to action. When I'm working with students, questions are raised where qualification of, and support for, a position or decision (to take or already taken) helps me get a sense of whether the person I’m talking to is qualifying or has chosen in an intellectual sense (I recognize each as requiring critical thought). In addition to the 'before' or 'after' aspect of deciding, I too find it important to know how their thinking is working or why they thought something through the way they did. Sometimes our goal is to find the best way, and how do we do that. Sometimes it is about not arguing with their assumptions as much as helping them determine if they have gathered enough background information so that their judgment has some legitimacy.
    I liked your point in the original post when you mentioned...”Lots of institutions put "critical thinking" on their to-do list. Good definitions should lead to good implementations and good assessments.” Committees leave Critical Thinking up on the to-do list because it can probably be assessed (not necessarily well), but I haven’t seen the mechanical blueprints of how to do it precisely either. My strategy then is to not assess critical thinking in a formal way, but to simply promote it. I implement by telling students to ‘judge away’, but make sure to have the determination to figure out how to judge well.
    Enjoy the long weekend and thanks for the reply.

  6. Thanks for the interesting comments, Dean, and the weekend wishes. Likewise! If you figure out a good answer to this knotty riddle, please let me know!

  7. Hi Dave,
    I was looking at this question of critical thinking (again) for some other work I’m doing and have some more thoughts when it comes to some of the questions you raised in your original blog post. Here’s a little thought experiment for you…What if we tweak the Delphi Report definition as follows:

    We understand critical thinking to be interpretation, analysis, evaluation, and inference, as well as explanation(s) of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which judgment is based, which results in purposeful, self-regulatory judgment.

    As you point out in the original post, using the original definition, ‘judgment’ is an activity with a particular type of result. Using the above reordered definition, judgment is a result, which I pointed out in my previous posts, makes more sense (to me at least). The original definition sets up an assumption analysis of what is the ‘action’ and what is the ‘result’, which in itself can be confusing.

    The other point I wanted to share centers on the exclusive use of iterative/recursive questioning to come to a conclusion. Upon some reflection, the ability to think recursively doesn’t mean we all have to march to the recursive drum (where everyone is interpreting, evaluating in lock step), which is important. We both know that once you are on that recursive drum loop, infinity soon approaches, which really gets in the way of looking for a better answer. At some point we need to substitute reconciliation for recursion, and maybe that is the aspect of critical thinking that needs a little more examination.

    In order to accommodate multiple judgments (even the self-inflicted kind ☺), the judge(s) compare reconciliation points and decide (that determinism thing again) how far along we the carry out the iterative process before it no longer serves a useful purpose (which your switchboard operator questions highlighted). My question is, do you think that reconciliatory thought should be included in your recipe for teaching CT?


  8. Dean, it's interesting that I was thinking about this sort of degenerative loop today, but from a completely different angle. I will have to wait for a more opportune moment to write it all down as a new post. Thanks for the comment!

  9. Hi Dave,
    Glad you are still working on the riddle. Maybe in your new post you can touch on how you think derivative analysis precedes the iterative analysis. I know this is like a prequel to something written way back in 2009, but hey, if we are going to solve this thing, we might as well work out how input/output change plays with everything discussed so far. I'm looking forward to what you have to say.