COOMs and peer feedback
Sunday, August 19th, 2012 06:13 pmI signed up for Eric Rabkin's Fantasy and SF course a while ago, partly to get some ideas for my own forthcoming fantasy course, partly to see how Coursera handle massively multiplayer open online courses (MOOCs), but mainly to have fun being a student and hanging out in the forums discussing fantasy and SF. The main thing I have learnt so far is how difficult it is to do course work on a tablet while travelling around Turkey (even the part of Turkey that tends to have Wifi in hotels). The tablet revolution is some way in the future, folks, and for once I find myself in agreement with Bill Gates: for school work you really want a good old-fashioned desktop computer.
I also learnt something about peer grading, and not just that it is a PITA to write feedback on four of your fellow students' essays on a tablet using a dodgy Wifi connection in a hotel lobby. What the Coursera experiment shows is that peer feedback (also called "peer review", but I prefer to reserve that term for academic journals) can be made practical, but it cannot be made reliable. The way it goes is that if you submit your essay by the deadline (a hurdle I failed to clear on my first essay due to the aforementioned technical problems) it gets sent to four other students, who write comments and grade it for "form" (style, grammar, organisation etc.) and "content" (basically whether it has an argument worth reading). If you fail to give feedback, your own grade is lowered (and indeed mine was, for the aforementioned technical reasons). It's actually a pretty neat system, but unsurprisingly there were plenty of complaints about it on the forums, largely from people who thought (often correctly) that their peer graders were as qualified to assess academic writing as they were to assess microcircuit design. If this were a proper credit course, then I too might have been as annoyed as some of the forum posters, but since I was doing it largely for fun, I just enjoyed bad feedback as entertainment. My favourite comment was that my essay read too much like a research paper and I would be better off finding my own voice. Unfortunately there is no way to give feedback on feedback; otherwise I would have pointed out that (a) in proper universities, essays are supposed to read somewhat like research papers, and (b) I write research papers; that is my own voice. I was also criticised for citing works the reader may not have read. Sure, when I'm reading a journal article and come across an unfamiliar citation, I put it down right away, just like I never follow a link unless it's to a website I've already looked at. Anyway, the experience has encouraged me to do more with peer feedback in my own courses but never to allow students to actually give grades.
The oddest thing, though, was the fact that some students actually plagiarised their essays, which has made Prof. Rabkin's course notorious (at least among the people who read The Chronicle of Higher Education). Why a student would plagiarise an essay on a course for which they get no credit is puzzling. I suppose there are a few who plan to put the course on their CV, but that doesn't seem a big enough incentive to plagiarise. It's not as though someone is going to give you a job because you took an online course in fantasy and science fiction. Some people suggested in the forums that it was the result of students from "other cultures" not really knowing what plagiarism is. I've come across this argument a lot, and I don't find it very convincing. Anyone who knows how to use the Internet knows that pasting a Wikipedia article into your essay is cheating, and not at all like the clever uncited allusions to classical writers that some "other cultures" encourage. Maybe it's just a habit students pick up in high school and find hard to shake, like shoplifting.
I also learnt something about peer grading, and not just that it is a PITA to write feedback on four of your fellow students' essays on a tablet using a dodgy Wifi connection in a hotel lobby. What the Coursera experiment shows is that peer feedback (also called "peer review", but I prefer to reserve that term for academic journals) can be made practical, but it cannot be made reliable. The way it goes is that if you submit your essay by the deadline (a hurdle I failed to clear on my first essay due to the aforementioned technical problems) it gets sent to four other students, who write comments and grade it for "form" (style, grammar, organisation etc.) and "content" (basically whether it has an argument worth reading). If you fail to give feedback, your own grade is lowered (and indeed mine was, for the aforementioned technical reasons). It's actually a pretty neat system, but unsurprisingly there were plenty of complaints about it on the forums, largely from people who thought (often correctly) that their peer graders were as qualified to assess academic writing as they were to assess microcircuit design. If this were a proper credit course, then I too might have been as annoyed as some of the forum posters, but since I was doing it largely for fun, I just enjoyed bad feedback as entertainment. My favourite comment was that my essay read too much like a research paper and I would be better off finding my own voice. Unfortunately there is no way to give feedback on feedback; otherwise I would have pointed out that (a) in proper universities, essays are supposed to read somewhat like research papers, and (b) I write research papers; that is my own voice. I was also criticised for citing works the reader may not have read. Sure, when I'm reading a journal article and come across an unfamiliar citation, I put it down right away, just like I never follow a link unless it's to a website I've already looked at. Anyway, the experience has encouraged me to do more with peer feedback in my own courses but never to allow students to actually give grades.
The oddest thing, though, was the fact that some students actually plagiarised their essays, which has made Prof. Rabkin's course notorious (at least among the people who read The Chronicle of Higher Education). Why a student would plagiarise an essay on a course for which they get no credit is puzzling. I suppose there are a few who plan to put the course on their CV, but that doesn't seem a big enough incentive to plagiarise. It's not as though someone is going to give you a job because you took an online course in fantasy and science fiction. Some people suggested in the forums that it was the result of students from "other cultures" not really knowing what plagiarism is. I've come across this argument a lot, and I don't find it very convincing. Anyone who knows how to use the Internet knows that pasting a Wikipedia article into your essay is cheating, and not at all like the clever uncited allusions to classical writers that some "other cultures" encourage. Maybe it's just a habit students pick up in high school and find hard to shake, like shoplifting.