Friday, July 02, 2010
Online Evaluations
In the spring we took the plunge and converted our end-of-term student evaluations from paper to electronic. We used the identical form otherwise--that was the only change. Looking at the results, it's obvious that the distribution of scores changed dramatically. The graphs below show the histograms for section averages. The number of sections is around 250 in each.
The bottom two are from paper distribution. The top one is the electronic, and shows a bell-shaped curve with a big spike in the middle. I assume that there are a lot of students who just bubble in 4s all the way down the sheet. That would explain the middle of the graph. But why would the skew in the paper version practically vanish online?
Subscribe to:
Post Comments (Atom)
-
The student/faculty ratio, which represents on average how many students there are for each faculty member, is a common metric of educationa...
-
(A parable for academic workers and those who direct their activities) by David W. Kammler, Professor Mathematics Department Southern Illino... -
The annual NACUBO report on tuition discounts was covered in Inside Higher Ed back in April, including a figure showing historical rates. (... -
Introduction Stephen Jay Gould promoted the idea of non-overlaping magisteria , or ways of knowing the world that can be separated into mutu... -
In the last article , I showed a numerical example of how to increase the accuracy of a test by splitting it in half and judging the sub-sco... -
I'm scheduled to give a talk on grade statistics on Monday 10/26, reviewing the work in the lead article of JAIE's edition on grades... -
Introduction Within the world of educational assessment, rubrics play a large role in the attempt to turn student learning into numbers. ... -
Inside Higher Ed today has a piece on " The Rise of Edupunk ." I didn't find much new in the article, except that perhaps mai... -
"How much data do you have?" is an inevitable question for program-level data analysis. For example, assessment reports that attem... -
I just came across a 2007 article by Daniel T. Willingham " Critical Thinking: Why is it so hard to teach? " Critical thinking is ...
One answer might be how you distribute the online version. Paper evals are usually done by students who are present on given day at the end of the term. Online evals are probably going to include students who are registered but who may not have attended regularly or at all. Such students are likely to rate the prof on the lower end, I expect.
ReplyDeleteI know you've commented several times on this particular article; I don't recall if you've said anything about comparable response rates?
ReplyDeleteApropos of student evaluations in general, I am interested in reactions to this story that ran in the Boston Globe on Sunday: http://www.boston.com/bostonglobe/ideas/articles/2010/07/04/what_happened_to_studying/
ReplyDeleteApparently a major study is about to come out claiming that students study a lot less in College than was the case a few decades ago (and this has been so in a period of grade inflation). The relevance of this to student evaluations is made in the last portion of the article, specifically through a quotation from Murray Sperber, a visiting professor in the graduate school of education at the University of California Berkeley: "Course evaluations have created a sort of “nonaggression pact,” Sperber said, where professors — especially ones seeking tenure — go easy on the homework and students, in turn, give glowing course evaluations."
This particular issue seems to me to be just the tip of the proverbial iceberg. The subjugation of an inherently elitist enterprise, education (from the latin for "to lead out" -- so there are leaders and followers), to a consumer-driven model has resulted in a variety of perverse affects. We, the professoriate, have participated for decades in selling college eductaion basically as a means of career and income enhancement. Of course, students' work in college has become much more focused on "the degree" rather than "the education." It's "the degree" that has been the ticket to better starting jobs and incomes. So students who learn less, but get "the degree" are getting what they are paying for.
On response rates--I remember that they were lower, but the number isn't in the IR report for some reason. I'll have to ask.
ReplyDelete@Brendan. That's a really interesting article (link). The part that got me was this quote:
Hours spent studying is not the end goal of an education, of course, nor the only way to determine if someone is learning or will land a job after college. Marks herself points out that employers don’t generally care about the content of job applicants’ classes; they’re more interested in whether an applicant graduated, was able to meet deadlines, and work within a bureaucracy.
Is THAT what we're doing?
The question, what is a college education for, is obviously important, but the answer is not obvious, and the answer depends very much on who is asking the question. I believe that for many students, education has been at least somewhat reified as the process necessary to obtain the credential that is perceived ticket to some career. A large part of the process is perceived to be irrelevant but nevertheless required. From the standpoint of business -- especially large corporate institutions -- I think some level of skills are assumed to be acquired by a degree holder. (Though the demands for accountability in part stem from the realization that this is not always so.) But I do think the business world does see a sort of socialization in getting the degree also: the willingness to tolerate what is perceived to be boring and irrelevant and nevertheless to reach the successful conclusion mirrors aspects of the tedium that also exists in the business world. Meeting deadlines and conforming to bureaucratic structures are as well. A person who can work under these requirements and "finish the job" is more likely a good employee. This is an important aspect of what a degree conveys about a person. (Those who opine that academia should be more like business might really consider how similar they really are.)
ReplyDeleteIt seems like if we want students to learn 'real life' job skills we should just put them directly in the workforce and then figure out what skills they lack. That's tongue in cheek, but this thread make me wonder. A friend just recommened a book called DIY U, which seems to have more to say on this subject. I will try to find a copy.
ReplyDeleteWhen I was teaching math full time we had a number of grads getting jobs at a big drugstore chain. I tried to find out what they liked about our graduates, and the answer I got was that they needed people in management who could do fractions, and they were pretty sure that math majors could....