Links on Learning

"The State of Science Standards 2012" maps out an analysis of pre-college science instruction in the United States.

Quote:

A majority of the states’ standards remain mediocre to awful. In fact, the average grade across all states is—once again—a thoroughly undistinguished C.

There are individual state reports at the site.


Next up is a high school student's ambitious "The education system is broken, and here's how to fix it." One of his complaints relates to the way analytical loading is done (see my previous post):

So, this causes students to go about the "textbook" skipping everything but the formulas, and then memorizing those. Then, when the test comes along, those who had time to memorize their formulas do excellent, and those that had something going on get low grades.

Then, as soon as they're done with the test, they put in all of their efforts into memorizing the next set of formulas and have nothing left from the last set that they memorized except "I love *whatever the topic is*, I got a 96% on that test".

 In a similar vein is a post from mathalicious.com: "Khan Academy: It's Different This Time." The author is critical of the eponymous video-based instruction site and its methods, claiming that:

Khan Academy may be one of the most dangerous phenomenon in education today. Not because of the site itself, but because of what it — or more appropriately, our obsession with it — says about how we as a nation view education, and what we’ve come to expect.

I think the author assumes too much about the implications of this. Video-based instruction is just a tool, which can be used correctly or abused.

A thread that runs through these three pieces is that intrinsic motivation is important. The report in the first link mentions the excitement that science generated during the Space Race, and how that's lacking today. The high school math student in the second article wants to know why, not just how. And the critique of the Khan Academy is alarmed at a potential "view and spew" pedagogy (my term, not the author's).

Learning terms, rules, methods, facts, connections, and so on can be pretty dry. This constitutes what I'm calling the analytical load required to do something more interested. Learning to play chords on a guitar is slow and painful, but then you get to play songs, which is fun.

I see tight, focused on-demand instruction like the Khan Academy as an essential resource for learning and reinforcing an analytical load. This can be augmented by additional material that motivates learners. There are plenty of ways to do that. Anything that looks like a story is good (history of science, for example). Applications that involve creativity are the ultimate objective.

All of the sources linked above are rightly critical of the prepare-and-certify model of education, which in practice turns into drill-and-test, with almost entirely external motivation. Teachers face a battle in winning back student enthusiasm against this machine. There's nothing wrong with being concerned about grades, but if that's all there is to it, students face a rude awakening after graduation.

More on that theme in an article at Common Dreams: "Wild Dreams: Anonymous, Arne Duncan, and High-Stakes Testing."

Finally, a link from Education Week on the subject of testing students who want to be teachers: "Analysis Raises Questions About Rigor of Teacher Tests." This is the meta-problem.

Gresham's Law

Well, not exactly, but close enough (from wikipedia):

Gresham's law is commonly stated: "Bad money drives out good", but more accurately stated: "Bad money drives out good if their exchange rate is set by law.".
This law applies specifically when there are two forms of commodity money in circulation which are required by legal-tender laws to be accepted as having similar face values for economic transactions. The artificially overvalued money tends to drive an artificially undervalued money out of circulation ^[1] and is a consequence of price control.

It seems like it needn't be an exchange rate set by law, though, in other contexts: it's the "being accepted at face value" that matters. For example, if a fake degree is accepted just as much as a real one, then who'd bother to get a real one? (Well, lots of people, but you see my point.) Sound ridiculous? Maybe you missed the story about the Pakistan MP scandal, where many were exposed to have faked their credentials.  Here's the quote (from ABC News):

The chief minister of Balochistan Province, Nawab Muhammad Aslam Raisani, told the press: "A degree is a degree, whether it is fake or genuine."

 What else is accepted at face value? Most proxies are, by most people. We generally don't really see what we look at (see Darwin's theory of sexual selection, for example). Here's a quote new to me by Diane Ravitch, now rather infamous for switching on No Child Left Behind. From the Philadelphia Inquirer:

[...] Ravitch notes, we find ourselves quite possibly on the way toward "a paradoxical and terrible outcome: higher test scores and worse education."

In other words, substitute test scores for authentic outcomes, and maximize to absurdity. Here's an opinion piece in Ed Week that describes the result: inflation of scores by any means possible. Here's the "debasing the currency" part:

Over the past several years, efforts to "hold teachers accountable" and "hold schools accountable" have produced perverse consequences. Instead of better education, we are getting cheating scandals, teaching to bad tests, a narrowed curriculum, lowered standards, and gaming of the system. Even if it produces higher test scores (of dubious validity), high-stakes accountability does not produce better education.

It seems like the policy makers have never heard of Murphy's Law either.

Open Courses

A technology article in The Chronicle yesterday describes the Obama administration's plan to develop open courses and give them away. This is a fascinating idea--the sort of thing that I mused about in the post "Unplanned Obsolescence" a few weeks ago. Carnegie Mellon's Open Learning Initiative is given as the model for this plan. The delivery of the course is potentially fully online, but example cited is a hybrid course with content developed by a group of experts and leveraging online capabilities for what I would call the low-complexity part. This is a natural way to optimize the delivery of instruction, and you already see it with course packs delivered from text-book publishers. Rote learning, or learning low-complexity modes of thought can be easily and tirelessly done through computer training. It's potentially more fun too. The Rosetta Stone language software is like this, using instant feedback, images and sound to train vocabulary (my wife used it to learn some Arabic). Where computers fail is at answering open-ended questions. If you've ever called your bank with an odd problem and had to navigate the automated phone tree (press one if you're still breathing...) you know how frustrating it is to try to solve high complexity problems with low complexity tools. So the need for actual instructors still exists, but their time could be applied more usefully to high-complexity tasks. This effect is noted in the article:

Carnegie's materials have already changed how Logan Stark's professor at California Polytechnic State University approaches her widely feared biochemistry-for-nonmajors class. Anya L. Goodman used to work from a prepared lecture, starting with the basics so she didn't lose anyone. Now she puts the burden on students to learn the basics online. She focuses class time on clearing up misconceptions, applying the materials to real life, and working in small groups.

The idea has merit, but there are certainly some big problems to overcome. If the courseware project is run and funded by the government, it may be open, but it surely won't be cheap. How, then do the course materials get updated? This isn't too big of a problem with, say, Euclidean Geometry, but for something like Finance, I imagine the books get re-written all the time. The article supposes that this might continue to be funded by the government, but this doesn't sound like a wonderful idea to me. It would be far better, methinks, if a culture evolved similar to the open source software movement. Rather than a set of "perfect" open courses designed and maintained centrally, a whole ecology of work collocated in some natural place--analogous to sourceforge.net--could grow and evolve, tagged with comments and other metadata. This depends on willing practitioners doing all the work. Faculty members taking the time to update old materials, probably. It seems unlikely on the face of it, but somehow it works for software. It works for Wikipedia.

A really interesting question is how course assessment ties into the courseware. Would it be developed and delivered in parallel, integrated with the materials? Or will assessment remain a second-thought tack-on for another decade? But if it is to be integrated, then the learning objectives need to be clear. There seems to be an opportunity for the assessment profession to get involved with this train before it leaves the station. Polish up your resume.

In a recent report I asked for, a group of twenty-plus institutions like ours had an average total expenditure on instruction-related items of 37% of total budget. We might suppose then that the asymptotic limit for reducing administration (assuming that all academic support is free somehow, libraries and such) is a 63% reduction in the cost of delivering programs. The cost of instruction could be reduced too, if the low-complexity components are off-loaded to technology. Moreover, competition in the fluid digital domain would tend to force prices down. I don't think it's unrealistic to estimate that a bachelor's degree could be delivered for about 25% of what it costs at a traditional college now.

Marshal Smith, senior counselor to the Secretary of Education seems to be the the guiding light behind this open ed plan. You can read his ideas if you have a subscription to Science in his article "Opening Education." You can also browse MIT's version of open courseware here.

Making Education Scientific

In my previous post I mentioned a good-natured debate on the ASSESS list-serv about whether education can be scientific or not. I focused more on whether the actual practice of assessment is scientific, but there are broader issues. One of them could be a meta-assessment of programs to see if they employ current knowledge about how learning works. That is, connect brain research to the delivery of instruction. As an example, an article in Science Daily from 2007 claims that physical movement enhances learning:

Kids asked to physically gesture at math problems are nearly three times more likely than non-gesturers to remember what they've learned.

Taking that at face value, and assuming that is works at the university level too, how long would it take this knowledge to seep into the practice of teaching college math?

On the other hand, what if one were to persue the practice of science to its logical end? Without rigorous standardization, it's very difficult to separate out potential causes from effects. Is such a rigid scheme appropriate or desirable for an educational setting? It would be a radical transformation. Such a transformational middle school math program is being tested in New York City, according to the website GothamSchools.org:

Students in the new pilot program [...] take a quiz every afternoon, and then receive a computer-generated schedule each morning, called a “playlist.” A student’s playlist might tell him to begin the day by meeting with a tutor, then to complete a set of online tasks, and then to work on a project with his classmates.

This robotic-sounding instruction method was concocted by the same group who came up with a rigid curriculum delivery described in a 2005 New York Times article, where this description of a teacher's work under the system is described:

But in his classroom, he was not designing anything; instead, he was following the balanced literacy script. In a 90-minute period, actual imparting of knowledge was restricted to a lesson as short as five minutes. Then pupils broke into small groups for independent guided work, and reconvened to share their efforts. School administrators made unannounced visits to ensure that teachers were using their rugs and abiding by the "flow of the day" schedule posted in each classroom.

This reminds me of fifth grade. For at least part of the year, our math lessons came from a purple SRA box. This program is now online, of course, but in the old days a student was suppose to start at the front of the box with division problems (there were probably other types of problems, but I remember only division). Our brains are not designed to do long division--an example of Moravec's Paradox, I'm sure. As a consequence, it's a hard slog to grind through what I learned later is basically an inverse convolution. (I now wonder if it could be done more easily by just inverting the convolution matrix... Have to think about that during the long meetings today.) At any rate, it was a real pain for an energetic fifth-grader, and probably turned many off to the idea of math entirely.

There was a system to the problems. Everything was self-scored, and if one did better on a particular test, the next one would be more challenging. The reverse was true too--if you missed many of the problems, the next set would be easier.

You can imagine what I did. I quickly discovered that if I "forgot to do" several of the hard problems, the next batch would be much less taxing. I shared this secret with my friend Ronnie, and soon we were working our way through the problem cards without breaking a sweat. While the other kids were trying out 14589/235, Ronnie and I were doing stuff like 256/16. Unfortunately he got caught. Fortunately he didn't rat me out, but I had to go back to the slog out of fear of being sent to the gulag.

The point is that systems can be hacked, cheated, and gamed. Any system. We hominids have faced that problem ever since people have lived in groups, probably, and have very good on-board apparatus for detecting cheats of all sorts. Computers and card systems aren't so good at it. Not yet, anyway.

Read what happens when a teacher in the teaching system described in the Times article employs a little flexibility and gets off track:

To avoid being caught if they did not follow the schedules, some teachers began "actually training their kids to switch subjects on command," [a teacher] says. "They can be doing a reading lesson, and if somebody walks through the door, all of a sudden they're doing the writing lesson."

This sort of side effect to standardization is inevitable. If we're going to go this route, it's imperative that we do the meta-analysis and ask: how can our beautiful system be gamed? It's the equivalent of building a new operating system to unleash on the world's computers. It needs to have safeguards against hacking, right? But no system is completely hack-proof, so in practice there's always a race between the hackers and fixers. (Computer people call malicious hackers "crackers," actually.) And this race, I think, is computationally so complex that it is out of the realm of prediction: that is, it's not a subject that can be easily made scientific.

Debating Science in Education

I've had a leisurely debate lately on the ASSESS listserv with Richard Hake about whether or not scientifically-based education is an oxymoron. In actual definition, of course, it is obviously not an oxymoron as defined in the 'pedia:

An oxymoron (plural oxymora (greek plural) or, more often, oxymorons)("sharply dull" in Greek) is a figure of speech that combines two normally contradictory terms. They appear in a range of contexts, from inadvertent errors such as extremely average, to deliberate puns like same difference, to literary oxymorons that have been carefully crafted to reveal a paradox.

A true oxymoron would be something like piano forte, which literally means soft loud. Since "scientifically-based" and "education" are in no way opposites, the phrase doesn't qualify. In common usage, however, people seem to bend the use of the word. In the present sense, it would be more direct to ask "can education be approached scientifically?" The answer to that question would be "yes" since anything can be approached scientifically. Science may not be able to tell us much about the subject, but we can attempt to use the methods of science on any subject we like. That, however, doesn't make for an interesting debate!

Dr. Hake takes the position that science-based education is not only possible, but happening. Here are the links to this thread. You can find current posts on this topic, including ones not cited below, here by searching for "scientifically-based".

• Original post
• My short response
• Richard's reply
• My longer response
  

In my last response I used a fact that I learned or was reminded of recently: that speech production and understanding happen in two different parts of the brain. They are called Broca's Area and Wernicke's Area, respectively. They're actually pretty far apart, but both on in the left hemisphere. In light of the creativity versus analytic thinking that I'm always going on about, this was obviously interesting. It also explains why I can understand far more German than I can produce, and suggests that the only way to get better at producing speech in a foreign language is to actually practice it. I'm sure all the language people know all this already.

The book this comes from is highly recommended. It's a short little thing I read on the flight back, called My Stroke of Insight by Jill Bolte Taylor. Jill is a brain scientist, who had a stroke in her left hemisphere one morning. The first-hand account of her brain deconstructing is fascinating:

[...] I was literally thrown off balance when my right arm dropped completely paralyzed against my side. In that moment I knew. Oh my gosh, I'm having a stroke! And in the next instant, the thought flashed through my mind, Wow, this is so cool!

The text is very readable, and describes the areas of her brain that were affected, and how she perceived those areas going offline. With the left hemisphere's speech centers debilitated, her right hemisphere took over:

In this altered stated of being, my mind was no longer pre-occupied with the billions of details that my brain routinely used to define and conduct my life in the external world. Those little voices, that brain chatter that customarily kept me abreast of myself in relation to the world outside of me, were delightfully silent.
[...]
As my consciousness slipped into a state of peaceful grace, I felt ethereal.

At the very least, this little book can make long committee meetings more bearable, by suggesting you how to tap into the mellow side of consciousness.

Another book that bears on the question of science and education tangentially is the current one I'm reading. Okay, it's a stretch, but bear with me. The book was sent to me by my historian friend Bob, after our discussion about the Russian Revolution. It's a classic called Darkness at Noon by Arthur Koestler and translated from German to English by Daphne Hardy. The pertinance to the topic at hand is the adherance to a pre-determined political position, despite facts on the ground. I would say it's a fair criticism of the accountability in education movement that it assumes we can assess far more accurately than we actually can the skills and knowledge of learners. And if that's a weak segue to this lovely piece of literature, then it's my fault and not Mr. Koestler's.

Here, the protagonist (and I presume the author) confronts the results of the Party's deep commitment to principles, to the ends justifying the means:

The cause of the Party's defectiveness must be found. All our principles were right, but our results were wrong. This is a diseased century. We diagnosed the disease and its causes with microscopic exactness, but wherever we applied the healing knife a new sore appeared. Our will was hard and pure, we should have been loved by the people. But they hate us. Why are we so odious and detested?

We brought you truth, and in our mouth it sounded a lie. We brought you freedom, and it looks in our hands like a whip. We brought you the living life, and where our voice is heard the trees wither and there is a rustling of dry leaves. We brought you the promise of the future, but our tongue stammered and barked. ...

This is another in long history of warnings that ideals and theories and principles and beliefs are useful only as long as they are fluid enough to conform to reality. A theory should be like a suit you use on occasion and then put back in the closet, not like a suit of armor to climb in to before viewing the world through its helmet's grating.

Blags and Electronic Resources

On the education subreddit I came across 100 Most Inspiring and Innovative Blogs for Educators. It has some good finds, including Jane's E-Learning Pick of the Day. Jane posts small articles with links to interesting online resources. Go look for yourself.

I found a couple of mind mappers I hadn't seen before, and one that a friend had mentioned called The Brain. The video demos look good.

There's also an open-source web-based HTML editor called Amaya I hadn't seen before.

By the way, the neologism "blag" comes from "blog" as an intentional misselling, courtesy of xkcd:

So...log comes from a ship throwing a piece of wood over the side to see how fast it's moving, which comes to mean the book the readings are written in, which comes to mean any form of recording information regularly. Meanwhile "world wide web" gets shorted post haste to "web," and the two words get put together to make "web log," which contracts to "blog" and now (if you're sufficiently geeky and want to be ironic) "blag." Isn't language fun?

A Learning Paradox

Assessing learning isn't just done in the education industry, nor is it even exclusive to biological organisms. The computer scientists have been trying to teach computers to think for decades now. To paraphrase a saying about fusion power, we might say that true artificial intelligence is just 20 years in the future, and always will be. There have been challenges. Knowing a bit about those challenges might inform the kind of wet-brain learning that we try to measure from across our desks in the institutional effectiveness building (What? You don't have a building yet?).

A standard test for machine intelligence is the Turing Test. Alan Turing established some of the fundamental ideas of computer science, helped crack German cyphers in WW II, independently derived the Central Limit Theorem, among other things. He was also persecuted after the war for being gay and ultimately committed suicide by eating an apple laced with cyanide. I highly recommend The Enigma as a biography.

Rene Descartes didn't believe in thinking machines. Quoting him from the Stanford Encyclopedia of Philosophy:

If there were machines which bore a resemblance to our bodies and imitated our actions as closely as possible for all practical purposes, we should still have two very certain means of recognizing that they were not real men. The first is that they could never use words, or put together signs, as we do in order to declare our thoughts to others.

This is essentially the test Turing considered. In 1950 he muses about something the he calls the imitation game:

I believe that in about fifty years' time it will be possible to programme computers, with a storage capacity of about 10⁹, to make them play the imitation game so well that an average interrogator will not have more than 70 percent chance of making the right identification after five minutes of questioning. … I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. [source]

First of all, note that in 1950, computers were massive glowing things with very limited capacity by our standards. For Turing to correctly guess that by 2000, hard drives would be measured in Gigabytes is flat out astounding.

The main point, however, is that Turing poses as the test of intelligence an interview. The question is simple: are we convinced that the other side of the conversation is intelligent?

You can try out Turing's imitation game yourself, no doubt running on computers with gigahertz processor clocks. Just browse to jabberwacky.com and start chatting. Below is an excerpt of an argument I had with the thing about what day of the week it is. Its responses are in bold.


There are practical applications to all this. It's not just that we might look forward to mechanical devices that can understand plain language and converse with us, but the Turing test protects us from the annoyances of the bad kind of machine. For example, 'bots' may automatically try to log into this blog and leave a comment with a link to some site the owner wants to advertise. Google tries to prevent that with CAPTCHA--hard to read images that one has to interpret in order to proceed. The one below is taken from the Wikipedia page:
It's easy for humans, but hard for machines to decipher these. This brings us to the learning paradox I advertised in the title: Moravec's Paradox. The Wikipedia describes it thus:

Moravec's paradox is the discovery by artificial intelligence and robotics researchers that, contrary to traditional assumptions, the uniquely human faculty of reason (conscious, intelligent, rational thought) requires very little computation, but that the unconscious sensorimotor skills and instincts that we share with the animals require enormous computational resources.

What the computer scientists would be the hard part was what humans find hard--manipulating symbols and doing logic. What they discovered is that the hard part is actually the kinds of things we learn without much effort--how to talk, walk, see, and judge our surroundings, for example. This is a fascinating result.

An evolutionary explanation is proffered. The idea is that the older the cranial wiring is, the harder the function is to reverse engineer. So sight and visualization, for example, are hard to reproduce with computers because they've been fine-tuned by hundreds of millions of years of evolution, and optimized to the point where it's effortless for most of us to see and understand our surroundings. (Not all--see The Man Who Mistook His Wife for a Hat.)

In terms of higher education learning outcomes, this carries some warnings. Standardization is an algorithmic approach to assessment, for example. This falls into the category of stuff that's easy for computers. Notice that Turing didn't suggest that we give a machine a paper and pencil IQ test. I think few people would be convinced of machine intelligence if all it did was get some vocabulary and logic problems correct (for example). He suggested instead a messy, subjective form of assessment for intelligence. The paradox suggests that some kinds of thinking are very old and likely difficult to assess with discrete methods, while others (the more recently acquired by the species) may be easier because they are more suited to a regular, algorithmic-type approach of judging success.

Here is a an example of a situation that requires intelligence. One might say critical thinking. See what solution you come up with. I got it from here.

According to a news report, a certain private school in Victoria, BC recently was faced with a unique problem. A number of year 12 girls were beginning to use lipstick and would put it on in the bathroom. That was fine, but after they put on their lipstick they would press their lips to the mirror leaving dozens of little lip prints. Every night, the maintenance man would remove them and the next day the girls would put them back. Finally the principal decided that something had to be done.

First think of your own solution, then read the 'official one.'

She called all the girls to the bathroom and met them there with the maintenance man. She explained that all these lip prints were causing a major problem for the custodian who had to clean the mirrors every night. To demonstrate how difficult it had been to clean the mirrors, she asked the maintenance man to show the girls how much effort was required. He took out a long handled squeegee, dipped it in the toilet, and cleaned the mirror with it. Since then, there have been no lip prints on the mirror. There are teachers, and then there are educators.

This story is funny because of the creative solution. I requires a deep knowledge of social norms, human emotion, and game theory to really grok it. This is the kind of thing that computers may be able to come up with eventually, and when they do and can prove it through conversation, we'll perhaps name them intelligent. It's also the kind of thinking that's very hard to assess with an algorithmic approach, for the same reason it's hard to program--it taps ancient and deep resources within our minds.

See Also: More Thoughts on Moravec's Paradox