Building a Teaching and Learning Technology lab

Thanks to our always-helpful government grants office, we were able to find money to develop a technology lab for teaching and learning. We brainstormed for a way to augment our existing installations, which already provide a lot of technology for course instructors. Given our new foray into online learning, lecture capture seemed the best bet--it's focused, manageable under our budget, and can give us an immediate payoff. It's also scalable in application from simple video recording to much more sophisticated productions. It will shine a light on our distribution system for electronic media too, and prompt infrastructure improvements that will have multiplier effects.

"Lecture capture" is of course a terrible name for it. Lecturing means literally reading, which is what our academic forbears did before there were enough books to go around. Like wearing those heavy black robes in the May sun, some ideas are ripe for overhaul.

There's a nice introduction by Educause here (pdf), if you want to see what they have to say.

What if we started from scratch?


It's not just the robes and readings that have seeped into the weave of the academic tapestry. Exposing young minds to the conceptual treasures of our disciplines is constrained by the combinatorics of time and space so that that certain optimizations are so standardized we don't think about them. Like having students sit for fifty minutes at a stretch in classrooms. What if instead we could engage them for ten minutes at a time on focused topics all day long? Another optimization is that all students in the class get the same linear flow of narration, and the discipline itself gets strung out into a one-dimensional string, codified into textbooks and syllabi that have the same linear style.

Imagine if the world wide web were like that: a single linked list of information, so that you had to start at the beginning and work your way forward link by link until you got where you wanted to go. It's a lousy way to organize rich information.

Sometimes concepts do have prerequisites. Math is full of it. But even in math there are optional topics, and opportunities to go into more depth even the simplest subject. If you want, you can think of the long multiplication process you learned in third grade as a convolution product, which can be optimized with Fourier transforms. One of my professors in graduate school (David Kammler) took an approach he compared to "Island hopping" as practiced in the Pacific theater in WWII, which consisted of learning certain important topics in depth and assuming that students could then make logical leaps to nearby topics, which could be skipped or treated as exercises.

Imagine instead of a linear trip through a discipline, we captured key ideas and used those as anchors for a network of related facts, concepts, processes, etc. One immediate advantage would be the ability to easily update the network as the discipline changes. Group evolution is back in vogue? Add a node and stick it in there. No need to fell a forest for the new edition of a book. Another advantage is that focused topics can be crowdsourced, as in Wikipedia, with experts in sub-sub-sub-fields filling in blanks ad infinitum. Other thoughts:

• interdisciplinary connections become easier to integrate naturally
• because learning happens in detail, it's easier to assess a small topic than a big one, allowing for faster improvement of the captures and related pedagogy
• custom methods can be applied more easily to particular topics. Think apps. When you get to the bit about singular value decompositions, you can download the stuff you need to see it in action, and then it can go away. It's modular.
• It creates a way for professionals to agree on something. A whole textbook? Never happen. But a five minute tour of Darwinian evolution? Better odds. This standardizes at the right level. You can still customize your curriculum--just match assessments to content.

All of this is probably being done in bits and pieces now, this divide and conquer approach. But it hasn't gelled yet into a common framework supported by real infrastructure and common culture (like say Wikipedia is).

I don't want to go all Utopian here, but let me push this one step further. With a rich network of high quality instruction/assessment/activities modules, the whole idea of a course becomes outdated. I know, I know, that's scary stuff. But if we don't have to have students sitting in desks for fifty minutes at a time, why do we need them for 15 weeks? All that really matters is that they learn and demonstrate mastery of concepts that we think are important. So then our whole bureaucracy can go hard a-port and do something really interesting. All that bricks and mortarboard infrastructure can be focused on motivating students. I've written enough about that already, so I'll stop here.

I should hasten to add that our modest project at my institution has none of these grand ambitions.  I will be selling the idea that a "lecture capture" doesn't need to be 50 minutes long, however, and keeping my eye on the horizon...

Part Four: Philosophy

Why Assessment is Hard: [Part one] [Part two][Part three]

On the ASSESS listserv, Steven Zerwas wrote yesterday to seek opinions about logical fallacies in assessment. This turned into a good discussion, and a line of thought for me was sparked by Joan Hawthorne's contribution, when she wrote:

[T]here's the intriguing bit of considering the arguments of people resisting assessment from their perspectives -- what are they really concerned about? Some just don't want to do what they see as "more" but I have known many who have serious and valid pedagogical and/or philosophical concerns.

A few posts back I asked where are the error bars? to encourage modesty in our (the assessment community's) claims about what we can and can't legitimately say about outcomes assessment and other statistical magic. The idea I'd like to unpack in this post is that "philosophical concerns" may be indeed serious and valid.

Yesterday I bought a third copy of Bertrand Russell's The History of Western Philosophy. I had given the first two away as gifts, and our own copy is buried in boxes somewhere. In any event, my wife made me give it back after I kept it with me in the car for a year as emergency reading material (when standing in line or eating alone). I had replaced it with Machiavelli's The Prince, but that's far to thin a book for real emergencies.

In his introductory, Russell gives his definition of philosophy, which is of interest to us here.

Philosophy, as I shall understand the word, is something intermediate between theology and science. Like theology, it consists of speculations on matters as to which definite knowledge has, so far, been unascertainable; but like science, it appeals to human reason rather than to authority, whether that of tradition or that of revelation. All definite knowledge--so I should contend--belongs to science; all dogma as to what surpasses definite knowledge belongs to theology. But between theology and science there is a No Man's Land, exposed to attack from both sides; this No Man's Land is philosophy.

The attacks Russell mentioned are launched from both sides at the unfortunate Assessment Coordinator. Claudia Sanny on the ASSESS thread notes that she has encountered the argument that "To do assessment properly, we must have a reliable and valid instrument." This is clearly an attack from the scientific perspective (although the actual motives may just be to delay the exercise, of course). On the other hand, we have committees and directors and other bureaucratic fauna at the highest levels of government applying pressure to produce comparable ratings for institutions on educational attainment of students. From the Spellings Commission Report:

The strategy for the collection and use of data should be designed to recognize the complexity of higher education, have the capacity to accommodate diverse consumer preferences through standard and customizable searches, and make it easy to obtain comparative information including cost, price, admissions data, college completion rates and, eventually, learning outcomes.

The leap of faith here is that comparable information about learning outcomes is achievable in any manner that resembles scientific endeavor. The exemplars given in the report (CLA, and NSSE, for example) are clearly not sufficient for the task at hand. This bold assumption about comparable learning outcomes is a naked belief. We may as well call it theology, to fit Russell's dialectic. In more practical terms, systems of belief are imposed by accrediting bodies often with unintentional irony (as when confusing standards-based and continuous improvement criteria [more]). This institutional theology to covers everything from the Dept of Ed to accreditation associations to our own exhortations to assess.

There is always a problem with passing such memes on to infidels. In religions, some combination of carrot and stick is usually effective. In the case of persuading a faculty member to give up a coffee with colleagues in favor of doing curricular mapping, we often have more sticks than carrots. It's hard to create true believers that way. We may get converts, as in the Spanish Inquisition, but they may not be very heartfelt. Worse, the belief structure may create more problems than it solves--problems like Blue Hat Syndrome.

Stephen Jay Gould liked the idea that theology and science were somehow mutually exclusive as magesteria (so far, so good), and that this obviated the questions that created conflict between the two. I just can't swallow the conclusion. There seems to be good reason why naked belief and predictive validity are natural enemies.

The source of the authority in a theology is We-Say-So (WSS). The source of authority in science is predictive validity. Which of these authorities do we invoke most often when persuading colleagues? This is an uncomfortable question because there is no easy way out here. On the one hand, a WSS convert is not likely to be a long term ally, but it's an effective way to get things done in the short term. On the other hand, hard science is essentially impossible to do in the context of teaching and learning as it's practiced in higher education. Between this Scylla and Charybdis lies Russell's definition of philosophy.

We can appeal to reason without the gold standard of predictive validity and without WSS. This means using politics judicously, telling stories effectively, and not straying far from obvious relevance to faculty. It means creating a language that is as jargon-free as possible, and using WSS only to get the conversation started, if at all. Reason and enlightened discourse are by themselves no panecea nor recipe for dealing with all compliance woes, but rather I suggest that there is a way between WSS and pure science that is likely more productive than either of those. This kind of thing was suggested by David Shinn on the listserv [here], who says it better than I have:

The answer? Listen. Find an ally or two. Cajole. Look for an opening to get your foot in the door. Patience. Small steps. And most important, when your institution finally does conduct assessment - use it! Then more patience and listening.

This seems to me a philosophical approach. I would add a comment that I heard from my conversations yesterday, from an assessment director: you need to be in the classroom practicing what you preach.

As a coda to all this, I'll relate a conversation I had with my dad last night. He retired after teaching many years of college math, high school, and finally middle school. He is humble man, and I take him at his word when he says that the last year of teaching, all of his students passed the state exam in mathematics, and two thirds of them exceeded the standards. He talked about reaching the conclusion that textbooks didn't have enough review built into them, and started a summer program of review for his students: two worksheets per week for ten weeks. The reward for completion was a pizza. Then he discovered the Saxon textbooks, which he convinced the administration to adopt.

But he was baffled by the WSS requirements from the state. Over time, they would create and then drop initiatives. Some were completed, most not, all involving lots of work constructing detailed rubrics and such. To his mind, none of it ever came to anything, and was a waste of time. To me, this is a cautionary tale. Good results don't come from theology. In my dad's case, they didn't come from science either--not really. It's easier in math to measure simple learning outcomes than in some other areas, but it's always hard to do controlled experiments (a point elaborated on by ASSESS listerv contributer Steve Ehrmann in his TLT article here). Reason and the human ability to deal with complex problems, when combined with good intentions, is sufficient to show results. Even when we can't demonstrate predictive validity the way the physicists can.

Next: Part Five