Creating Genius, Part II

Last time, I commented on Richard Hamming's talk to Bell Labs in 1986 entitled "You and Your Research." Let's continue.

He has some interesting things to say about creativity--that mysterious source of new ideas. He gives some advice that will seem familiar to any mathematician:

If you are deeply immersed and committed to a topic, day after day after day, your subconscious has nothing to do but work on your problem. And so you wake up one morning, or on some afternoon, and there's the answer. For those who don't get committed to their current problem, the subconscious goofs off on other things and doesn't produce the big result. So the way to manage yourself is that when you have a real important problem you don't let anything else get the center of your attention - you keep your thoughts on the problem. Keep your subconscious starved so it has to work on your problem, so you can sleep peacefully and get the answer in the morning, free.

This seems to be a common phenomenon. For me personally, I've had this experience many times when programming or doing research intensively. When it happens, there is a real sense of breakthrough--of understanding something you didn't before. Not every time is it sustained--sometimes my subconscious gives me the wrong answer. But more often it's correct, and mysterious about how it got there.

Seeding creativity is important. "You can't always know exactly where to be, but you can keep active in places where something might happen." Hamming means looking out for the important problems.

If you do not work on an important problem, it's unlikely you'll do important work. It's perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them.

Part of the seeding comes from having an open door, Hamming says, both as metaphor and literally. Tolerating distractions is the price of staying in the conversation.

Taking the big view is also important.

You should do your job in such a fashion that others can build on top of it, so they will indeed say, "Yes, I've stood on so and so's shoulders and I saw further.'' The essence of science is cumulative. By changing a problem slightly you can often do great work rather than merely good work. Instead of attacking isolated problems, I made the resolution that I would never again solve an isolated problem except as characteristic of a class.

Now if you are much of a mathematician you know that the effort to generalize often means that the solution is simple. Often by stopping and saying, "This is the problem he wants but this is characteristic of so and so. Yes, I can attack the whole class with a far superior method than the particular one because I was earlier embedded in needless detail.'' The business of abstraction frequently makes things simple.

This resonates with me, and that is probably a sign of my own peculiar weakness. I don't like complicated details, doubly so when they seem irrelevant to the problem at hand. My poor brain is always looking for the simplest way to do something. Sometimes that's a good idea, and sometimes not. Remember that generalization is a type of inductive reasoning. Some characteristics of such modes of thinking are that:

• It is creative--there is no general procedure for producing general procedures
• There is no guarantee of success. It takes persistence to get anywhere in this trial and error process.
  
• It requires a knowledge of the analytical tools necessary to check to see if your solution is correct. If you don't know right from wrong, you won't get far.
  

For many types of problems, this means adopting or creating a philosophy that makes sense--one that can incorporate the peculiar nature of the data and processes you're working with. For example, Darwin's Big Idea was that there had to be something in the reproductive process that acted like genes. He didn't know about genes, and it gave him fits because the evidence wasn't there (Mendel's work wasn't well known until later). But he arrived at the right philosophy. Leucippus and others figured out a good general way of thinking about matter as atomic.

Think about your curriculum. Think about the big problems you deal with in your job. How often do you go back and think about the big picture--the most general way of thinking about these issues? In the curriculum, we typically immerse students in detail and fail to emphasize philosophical underpinnings (in my experience). As an example, a liberal arts curriculum is typically a list of stuff. Writing, quantitative skills, etc. are assembled in registrar's lists with the assumption that some whole comes from the parts. What is this gestalt? Is it to enable students to have a grounding in the big problems of our time? To give a personal answer to the meaning of life? To have the technical means to begin to think generally (inductively)? To prepare them for a major or for the job market? Or just to fulfill accreditation requirements? The big WHY of gen ed easily gets swamped in details and lost. Even if we pay lip service to a mission statement, it's very difficult to actually integrate that philosophy into course work and classroom activities to instantiate it.

Alter the problem. Instead of asking how much Stanislav learned, ask how much do we think Stanislav learned. The first problem is probably impossible to answer; the second is trivial. And yet, the second is the more important question. When Stanislav goes out into the big wide world, his supervisor, teacher, or mentor will form an opinion about Stanislav's capabilities and act on that opinion, not some standardized test score, however accurate it is advertised to be.

Hamming concludes with an Apollonian "know thyself."

If you really want to be a first-class scientist you need to know yourself, your weaknesses, your strengths, and your bad faults, like my egotism. How can you convert a fault to an asset? How can you convert a situation where you haven't got enough manpower to move into a direction when that's exactly what you need to do? I say again that I have seen, as I studied the history, the successful scientist changed the viewpoint and what was a defect became an asset.

The idea that defects and limited resources produces creative solutions was mentioned in the first part of the article. Perhaps these limitations spawn conditions to generalize and ask WHY, simplifying and redirecting us to the philosophical underpinnings of the problems to be solved.

Creating Genius, Part I

In March of 1986, Richard Hamming gave a colloquium seminar talk to Bell Labs scientists. You can find the text of the talk, which was entitled "You and Your Research" here. I think it holds valuable insights for anyone seeking answers. That's all of us, right?

Hamming is famous for his results in communications (e.g. Hamming Codes), and he had the opportunity to work with and observe some of the greatest scientists of the 20th century. He relates that working at Los Alamos with the likes of Richard Feynman made him envious. In his words, "I saw quite a few very capable people. I became very interested in the difference between those who do and those who might have done."

This is a question that's in the air, of course. The recent book Outlier by Malcolm Gladwell takes up a similar question, and I've blogged about the topic in the past. Hamming's remarks have special weight because he did first-class work himself.

There are several themes that Hamming employs. On confidence and courage:

Our society frowns on people who set out to do really good work. You're not supposed to; luck is supposed to descend on you and you do great things by chance. Well, that's a kind of dumb thing to say. I say, why shouldn't you set out to do something significant. You don't have to tell other people, but shouldn't you say to yourself, "Yes, I would like to do something significant.''

It's not really about luck, because people who do good work often go on to do more good work. Think of Feyman or Einstein or Darwin or Newton... I'm focusing on scientists because that's what Hamming is directly addressing, but you can create your own list in other fields of endeavor.

His second point is interesting because it speaks to the whole idea of assessing learning outcomes, if obliquely. Hamming says there are different ways of measuring brains. As an example, he talks about an employee:

I met him when I was working on a problem with John Pierce's group and I didn't think [the employee] had much. I asked my friends who had been with him at school, "Was he like that in graduate school?'' "Yes,'' they replied. Well I would have fired the fellow, but J. R. Pierce was smart and kept him on. Clogston finally did the Clogston cable. After that there was a steady stream of good ideas. One success brought him confidence and courage.

The ability to do good work is more than the cognitive skills we typically try to measure, in other words. We probably ignore a lot of what's important when we focus on narrow learning outcomes, rather than considering a more holistic approach.

Other factors that Hamming cites as important are age, and working conditions--bad ones are good because they spark innovation! Embedded in this rich piece is a theory of learning as an exponential process:

"Knowledge and productivity are like compound interest.'' Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity - it is very much like compound interest. I don't want to give you a rate, but it is a very high rate. Given two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime.

This is very similar to the conclusions that others have come to (see Gladwell's book, or my previous blog posts citing other studies). But effort and drive are not enough--it's key that they must be directed. A darker side of all this is that other things have to be neglected in order to accomplish it.

Tolerating ambiguity and looking out for evidence that contradicts your current understanding is another recommendation from Hamming. He quotes Darwin on the topic, saying that "Darwin writes in his autobiography that he found it necessary to write down every piece of evidence which appeared to contradict his beliefs because otherwise they would disappear from his mind." Stephen Jay Gould used this idea as a foil in his books, often referring to the "exception that proves the rule."

[to be continued]