Blue Hat Syndrome

I was at a state consortium meeting yesterday with administrators who have institutional research responsibilities, and the conversation naturally turned to accreditation and the on-site team that does the dirty work. It's a familiar theme at conferences too: the rogue team member who arrives on campus with something akin to religious zeal, and proceeds to demolish your carefully laid argument for compliance with SACS 3.3.1. I have such a story too. Probably everyone does. But hearing such grievances yesterday made me remember a hot summer in Kansas in 1982 at Fort Riley, as it turns out.

I had joined the Army National Guard and ROTC for two reasons. One was to help pay for college, but the more important reason was that my two best friends were doing it. And after a summer in Fort Bening, Georgia crawling around in the sand, I was back for round two as an officer cadet in the equally hot, equally humid midwest.

It's natural that uncertainty breeds expertise. Think of patent medicine 'cures' or soothsayers guaranteeing answers to questions. In the absence of compelling evidence, experts pop up like mushrooms. One such expert was Cadet Kane. From my journal:

On a cloudless hot day I perched on a wooden bleacher to listen to a green man talk about machine guns. The topic of his lecture sat on a wooden table on a tripod. He had spoken the same words every day for a lot of days, and they came rolling out of his mouth polished and round, like the stones you find at the bottom of a creek.

He seemed to never stop for breath, and the combination of his sing-song voice and the fierce sun made staying awake a chore. I pinched myself every few seconds. I hyperventilated. I held my eyes wide open for as long as I could without blinking. Nothing seemed to work, but I managed the appearance of wakefulness.

When it was over, my platoon broke for lunch, and I was able to talk to the other trainees for a bit. One of them was a military genius by the name of Cadet Kane. He gave me one of those "I know something you don't" looks.

"You can't shoot the M2 at people," he said. The M2 was the Army's unimaginative name for the gun we'd been hearing about. It's a monster that shoots bullets a half-inch across. His statement didn't seem right to me. What good is a machine gun if you can't shoot it at people, I wondered.

"It's a violation of the Geneva Convention," he continued. I refused to give him the satisfaction of seeming interested, so I encircled my mashed potatoes in preparation
for a desperate assault.

"Fifty calibre is too large a gun to be used for anti-personnel purposes. You can only shoot it at equipment."

That seemed ridiculous to me. I hadn't joined the Army to become a lawyer! What kind of morons had been at the Geneva Convention, anyway, I wondered. They were probably too busy boozing to get any real work done.

"But you CAN shoot at uniforms!" Cane announced gleefully. "Boots, dogtags, anything like that."

That was too much. "You mean to tell me that if an enemy soldier is wearing clothes, you can shoot him, but if he's completely naked, you can't?" I asked, giving up on my
potatoes.

"Not with the M2," he chortled.

I imagined my position being overrun by a hoard of nude men as I watched behind my lethal, but prohibited, machinegun. What would I do? Violate the sacred Geneva Convention, or let them run on by?

"Although," he said thoughtfully, "they might be wearing contact lenses, and there would be no way to know until they were very close. I'd say you'd have a good case for shooting them anyway, just in case they were wearing contacts."

I figure that the guys with machine guns will ALWAYS find a loophole. Keep your clothes on. Mark Twain was right: naked people have little impact on society.

This legalistic notion is, I think, urban legend, and generally such expertise can be easily disputed by the likes of snopes.com. But there is plenty of snake oil still to be sold anywhere uncertainties and stress are to be found. I related this instance because it's personal and funny, and because it segues into the second half of this tale nicely.

The blue hats were symbols of leadership during training. The officers in charge rotated the cadets through positions of authority like commanding officer, executive officer, platoon leader, and so forth. The idea was that the trainees would get some experience in leadership positions by having a day on the job. When it was your turn to be executive officer, you'd get a blue helmet with 'XO' painted on it. This contrasted nicely with all the ordinary green helmets around, and looked a bit like the UN had dropped in to observe us. But I found out quickly that to be so annointed with power, temporary as it might be, did not make one a peacemaker. Au contraire.

It became quickly apparent that when a perfectly nice guy or gal (this was coed training) donned the blue helmet, he or she became an instant jerk. I don't mean the kind of crankiness that might be natural when you've come under sudden stress, but rather all-out general jerkishness--yelling at the other cadets, chewing them out in public, strutting around like a peacock, and worse. It's as if Clark Kent became General Patton for an afternoon.

I resolved that when it was my turn to wear the blue, I'd not change my personality. I was inoculated, you see, by seeing the experience of others--this lycanthropic transformation into rabid drill instructor mode.

I failed. As soon as that blasted helmet was on my head, I became as bad as all the others I'd seen. This was not evident to me in the glare of the focus of sudden attention garnered by the 'executive officer' status of Cadet Company Bravo. I was too busy making plans, assigning tasks, worrying about the weather and the food, and what I'd say to the (real) major who commanded our group. But when it was over, I was able to gain a certain perspective and felt ashamed by own transformation--made doubly worse because I'd seen it coming. I wondered for a long time if this sort of thing was inevitable, or if it could be overcome, like exposing a werewolf to ever increasing doses of moonlight to control the disease. I called this phenomenon of sudden authority leading naturally to anti-social behavior as the Blue Hat Syndrome (BHS). I've seen the syndrome exhibited many times since then--including a couple more times in my own behavior.

So to tie together the strands of this story, imagine a blue-hatter who finds himself in possession of momentary power over others--the authority to review work and render judgments--and that the domain of interest is a fuzzy one where real expertise is not available. An example would be, say, the Spanish Inquisition. Another would be accreditation visits for determining compliance with learning objectives.

Uncertainty, as we've noticed, can easily generate a body of 'knowledge' that has little to do with reality, such as with astrology. For example, much of what is 'known' about assessment of learning falls into the same category. We imagine we know more than we actually do, because of the nice theories we've made. We speak of learning outcomes 'measurement' when it clearly is no such thing, and produce nice-looking rubrics and reductionist definitions that would warm the heart of any positivist, but which may have little applicability to the actual problem. Not all problems can be solved through such methods, but they're familiar and seem obviously destined to produce results until one actually tries to do so. This is a bad situation for the ones being reviewed.

The recipe is this. First, we have a good candidate for blue-hat syndrome on campus--someone who is probably persecuted on his own campus for ramming effectiveness planning down everyone's throats--suddenly able to see with sparkling clarity the way things really ought to be. He's prepared with flowcharts and Nichols diagrams spelling out in engineering-like precision the 'conceptual framework' carrying the deterministic guarantee of spiraling success. If only the practitioners at the institution would have the will and perspicacity to carry out this plan.

But they don't. This is because the magic plans and guarantees of success are just markers in a wide plane of possibilities, none of which is certain to lead anywhere useful. Total quality management is a bad fit for an educational enterprise because it's very difficult or impossible to certify what exactly a student has learned, and whether or not it is due to our efforts. Good leadership, vision, and judgment should be more prized than matrices and statistics, but that's not the case. Into this corridor between theory and reality strides our blue-hatted hand of fate.

The theory is seductive. Matrices and taxonomies and definitions and rubrics. It looks like science, and it could be if we were talking about measuring the expansion of the universe by counting up supernovas. But we're not. In the nebulous realm of assessing education, too much weight is given to the accouterments of science, so that it can easily become pseudo-science. But it is the outcomes of science that we are held accountable for. If we cannot show that our 'scientific' process produces results, we find the baleful gaze of the blue-hatted messenger falling upon our IE reports in disdain.

The situation is almost guaranteed to be a disaster. It's a recipe for reports that pretend to do what they're 'supposed to', to a tacit understanding that there is a game to be played, and that the most important factor in accreditation is who is on your visiting team. This is too bad, because the goal is a worthy one. As Yogi Berra is reputed to have said:

In theory, theory and practice are the same. In practice, they're not.

My own accreditation horror story is thus. The team member judging effectiveness was clearly affected by BHS. He exhibited the classic symptoms of self-inflation and utter certainty about his mission. Our plans were a mess, and we deserved to be gigged, so I can't fault him for that. But not actually reading our stuff was an insult. Rather, he took one look at our system and pronounced it unfit. He drew for me the 'correct' way to do institutional effectiveness--a flowchart to success. Our theoretical underpinnings, you see, were not of sufficient pedigree to pass the white-glove inspection. There was dust all over our section 2.5 in his judgment. But what really got me was his description of a real event. In his zeal to explain how things were supposed to work, he chanced from the beautiful theory into the ugly practice--his own.

The reviewer's college was a specialized one. They had an orientation course on this specialized subject for first-year students, to introduce them to the culture and prepare them for the subsequent curriculum. To find out whether or not knowledge from the course was retained, the students were tested in their senior year on this subject matter. My blue-hatted teller of this tale leaned close to make sure I got this point: the seniors weren't doing so hot on this test, he explained. He waited for the light to dawn in my eyes that this was a matter of institutional effectiveness. Then he went on, describing how they'd researched this problem carefully, figuring out how those test scores could come out better. The conclusion: teach this specialty course in the second year instead of the first. And you know what? It worked--the seniors scored higher after that change was made! The zeal practically dripped from his trembling lips as he concluded his description of the case study--this examplar of institutional effectiveness. I just love this stuff, he said.

Maybe he was right. Maybe his system, with its glossy flow of reports, was better than our system. But what's certain is that the system for accreditation itself--of marrying a BHS situation to a flawed epistemology--is one guaranteed to create not a culture of assessment so much as a neo-scholastic culture of procedural adherence and nit-picking.

Update: A friend emailed me an appropriate quote attributed to Kurt Vonnegut. "Be careful what you pretend to be, because you are what you pretend."

Update 2: See Dostoevsky's description of BHS from House of the Dead here.