Monday, October 4, 2010

Peer Review Ponderings

... or problems? pontifications? I'm just really liking my alliteration lately, but since "postdoc" also begins with "p" I may be wearing people's patience thin. Hee. See what I accidentally did there?

So as you know I have been grading. Oh how have I! It is generally accepted that the "peer review" process of students getting into small groups to read over and critique each others' papers is an important and useful part of the composition class process. I build it into my classes in such a way that my students take it seriously and look like they are working hard at it. And then I see the results and despair that they are actually getting anything from it.

How can you comment usefully on someone else's paper when you yourself are almost completely incapable of writing one? Yes, I know that they are learning important things that are not directly related to producing quality essays: to wit, that a comp class is all about practice and familiarity and that they are getting models of student prose and practicing evaluating it, as well as modeling the process of being self-reflective aware writers and we hope that this process will eventually be internalized in relation to their own papers, but still it is hard to look at a crap paper with almost no comments on it except for a few "good!" and "I liked this paragraph" and "maybe this is to short bc it is only 2 pages lol" and some random squiggly underlines. Le sigh.

And it is true that quite a few of them have already had some harsh object lessons about consequences and the importance of taking detailed instructions seriously. Peer review is often a good push to get the nonserious students out of the class, the ones who only show up half the time and were probably on their way to drop anyway, and a Look of Death plus sending them home for not having their papers or returning their peer evaluations can do a lot to thin the herd. Unfortunately I still have a few (all dudes with 'tudes) that turned in about a page and a half of lol-speak that doesn't fulfill the assignment any way. My time is a precious commodity and my struggling but eager students will suck it all up; I'd rather have the lost kids drop before the paper is in and give me that much more breathing room.

(Btw, why is it some students turn in stuff that is so clearly not trying? Are they afraid they will be good at school? Are they afraid they are not good at school and therefore want to protect themselves by pre-emptively writing crap that they can turn into a big joke later? Maybe they are ambivalent about being in college or about somehow betraying their family by going to college --- I certainly have a lot of first-gen students here and could see at least some of them getting hassled rather than supported by their parents. I just keep thinking if I could figure out what was driving them, I (we?) could figure out how to get through to them.)

Anyways I blew through the peer reviews today and wondered if I should shift them from revision to editing ---- they can't find a thesis in someone else's paper (why do they all keep marking the first sentence of the essay? is some high school teacher explaining that is where you put it?), but they can do pretty good at suggesting a title, formatting fixes, grammar mistakes, the occasional missing topic sentence, stuff like that. Of course, I am trying to reinforce the notion that revision is bigger and more important than editing, but that then leads to worksheets that are empty or that say nothing even when they are filled out and not very much useful for students to work with. Hmm.

And then there is the issue of groupings ---- do you have the blind lead the blind or put the blind with the one-eyed? Do you spread out the problem students or leave them all to sink in one group? I was hoping to have the stronger students be shining examples for the weaker ones this time around, but then you have the problem where the weaker student population and the slacker/flakey population overlap and suddenly each group has only half the number of responses to turn in with the final paper. The whole logistics of getting students to take stuff home and then bring it back and give it to someone else is just a nightmare, too.

What do you all do with peer review groups? Any magic tips that will instantly, painlessly make students into good writers with the magic of peer review? Please?

12 comments:

Fie upon this quiet life! said...

I've done peer review a couple of ways. Right now, because I have 4-hour classes one night a week, I am having the students read their peer's paper in class and have them do one and only one thing - ask for more details. These students are supposed to be writing 8-10 page essays every single time. And many of them have trouble filling out that much space. So I ask the peer reviews to find places where there is a deep need for more examples/details/explanation, etc. And it really, really was helpful for this last set of papers. That's not always going to be helpful in every instance. But for these long papers, yes! I'll keep doing it.

For my warm-and-fuzzy school comp class, I had the students exchange papers in the class BEFORE we did peer review. They read the papers at home and commented. Then they had a half an hour in class to discuss the comments and the papers with their peer. That seemed to work out well, too. But these were very sharp students who didn't need a ton of coaching overall. They were able to ask meaningful questions about content and again, seek for details.

Anyway - that's what I do, for what it's worth. As a writer, I really love getting feedback from other writers, but I know students have some issues with it at times. It's hard sharing your work.

Oh, p.s. I also give them points for peer editing as long as they "do a good job" - whatever that means - and have a fully finished rough draft to exchange. Usually that means that the people without drafts get zero points. So just showing up with a draft is like points for free. But not showing up with the draft is really a downer on the grade.

Sapience said...

My program has a very particular way of doing peer review, and it's different enough we don't even CALL it peer review anymore. We call them Small Group Tutorials. I don't know that you could do this sort of thing anywhere else, but maybe there's something you can adapt to your situation, because these work stupendously well with most levels of students, I've found.

Students bring a draft of their paper, with enough copies for everyone in their group (usually four students per group, sometimes five) and a copy for the instructor. They take the papers home, read them, and comment on them using a worksheet. The worksheet varies from class to class, but it can be anything from the "find the thesis" sorts of things to questions based on Joseph Harris' book Rewriting. (Which, by the way, I find really useful for getting students to think about revision in bigger ways, as well as getting them to use sources in interesting ways.) Students know from the outset that these responses to their group will be graded (they have to bring two copies of the worksheet--one for the writer, one for the instructor).

The next week of class sessions are canceled in favor of each group meeting for a one hour session with the instructor. The instructor leads the session, getting students to talk about their work in ways that go beyond "this is good" and "I don't understand this." They help students understand the difference between what sort of comments are helpful and which ones aren't, and making sure the big issues in a draft are actually addressed.

I'm going to semi-out myself by pointing you to the materials on small group tutorials that my program has on revision and tutorials. Hope you can find something that helps with your particular issues!

Bardiac said...

Wow, Sapience's program sounds great! Good ideas there.

Here's my little contribution. I change groups around. The first go is random. I grade peer revision, and students respond to their peer revision.

The second paper, I split the students into three rough groups, based primarily on their peer editing grade, and try to balance the groups so that each has someone strong, someone okay, and someone weak. Most of the students improve a lot at taking peer editing seriously between the first and second papers; I regroup for the third paper with the same system, but different groups.

For the biggest paper, near the end of the semester, I split the students into groups again, and put the strongest peer editors together. That's their reward for working hard, and also gives them a chance to give and get the best help on the most important paper. Most students are doing pretty well by that point, so they're getting pretty good help. Students who've been consistently weak or who haven't shown up are grouped together.

My system doesn't build a sense of teamwork as much as if they worked with the same folks through a semester, but it works well in other ways.

Belle said...

I've found that modeling it explicitly is wonderfully useful. And pointing out not only what I'm doing, but what I expect from them as I'm modeling it. Students feel vulnerable enough as it is - so I might use a paper I myself crank out in a few minutes. I then walk them through the process. It's well worth the class time and students respond with better p/r. It's pretty basic, I know, but some students need repetition or a first experience. And too frequently, that's my class.

Oddly enough, when I've passed my own stuff to colleagues for peer review, I get darned few useful things back. It's one of those skills we're supposed to know, but few actually teach!

Anonymous said...

The above suggestions are all great, so I won't repeat them :) One thing that I find helps with peer review that hasn't been mentioned is that I tend to have students do it only after they've received some short assignments back with extensive comments from me (assignments that I call "ungraded" - which count for credit but not for a letter grade). This gives them a sense of exactly how tough on them I'll be, and it makes peer review seem a lot more important to them, and it makes them see that "good work" at the top is only hurting their classmates. Also, I have my students free-write for 5 minutes before they start peer review on peer review days about the positives and negatives of their papers and then specific things they'd like their reviewers to pay attention to. This is helpful, too, because I have my students include a "writer's note" for me with their final drafts that asks them to do basically the same thing, and so this is practice that helps them learn how to communicate about their writing, and it makes them feel more ownership over the peer review process.

Horace said...

I've done group peer review as Sapience has, and while it is time intensive, it is effective.

Barring that amount of time and wiggle-room in a syllabus, I tend to build draft worksheets that have students identifying components-thesis, topic sentences, evidence, citation, analysis of evidence--first, and afterwards merely making three concrete suggestions.

I've found that simply the work of identifying what SHOULD be there is prompt enough for both students in a workshop pair. While it's hardly the most innovative approach, it has been a system that I've found (if students are going to revise seriously at the first year level) helps them to figure out what's missing and what needs adding on. After that, maybe an additional concrete suggestion or two is all that they can meaningfully handle. Email me directly if you want a sample of the sort of worksheet that I use.

Shane in SLC said...

This is off-topic, but I'm curious... Before classes began, you wrote:

I worked at a place with similarly unprepared students and have pre-existing lesson plan stuff at hand!

To which I replied:

Even without knowing exactly where you are, I'm skeptical that this is true. Prepare yourself to be shocked at how much worse your new students' preparation for college-level writing will be.

Was I right? I ask not to gloat in an obnoxious told-you-so manner, but because I'm genuinely curious whether California public schools really are superior to public schools in the South, as I strongly suspect (having attended Southern public schools myself for my primary and secondary education, and having taught at a third-tier Southern university not so long ago). Or was the writing you encountered from first-generation college students in Cali just as bad as what you're struggling with now?

Tree of Knowledge said...

Another thing that you can do is talk about what happens in an effective group before hand and have the students develop peer editing rules for the class (that they then have to follow). Then follow up with a group evaluation where they have to decide on at least one thing to do differently next time to improve the group work/editing process.

But the biggest thing is to just them get comfortable enough with each other to say things like "I'm confused--what are you trying to do here?" to each other. And the only way I know how to do that is to have them work with their groups/partners a lot before hand in non-workshop/editing groups.

Sisyphus said...

@ Shane in Utah --- you may remember I haven't taught writing at the UC level, just lit. (and comp at the CC level)

My master's degree was not in California but from a state school in a very poor rural area ---- and yes, it's a very similar population here. These students may be even a little bit *more* prepared, as I don't have the problem of large numbers of hostile 40-year-olds returning to school but unwilling to do any work or listen to someone younger than them.

On the other hand, man, did I suck as a writing teacher back then! No experience being in a writing class, no training, that one sucky "pedagogy" class met on Fridays during my first semester and all semester long we went over stuff *after* I had already done it for that week (what's peer review? Oh, I already had to do peer review and collect papers ---- yeah, that advice would have been helpful before hand!)

So I have no idea if those students would similarly have had trouble revising and improving because we did almost no writing in that class, just discussed abstract theoretical essays in a loose way in class every session until I got the essays, which I then failed. Those poor damn shortchanged students.

Sisyphus said...

Oh, p.s. I also give them points for peer editing as long as they "do a good job" - whatever that means - and have a fully finished rough draft to exchange. Usually that means that the people without drafts get zero points. So just showing up with a draft is like points for free. But not showing up with the draft is really a downer on the grade.

Yup, I do that. Don't have the length of time in class or the length of the papers to do the other bit.

tudents bring a draft of their paper, with enough copies for everyone in their group (usually four students per group, sometimes five) and a copy for the instructor. They take the papers home, read them, and comment on them using a worksheet.

I don't get it; that's a pretty standard peer review/conference format. I'm only doing the conference part one time because the department mandates at least one individual or group student conference, and I don't get paid enough to run through that with 75 comp students multiple times. I'm already dreading the one required time.

From the looks of the Harris book on Amazon preview, my students are not at that capability. I'm really reaching over their heads in making them respond to and quote texts, which is not at all the dept. norm --- the lecturers I talk to are doing personal essays until the end of the semester, no quotes.

The first go is random.

I totally should have done this --- I made groups based on their response paper quality so far, and then had a total change-up when lots of them didn't show or came late and I ended up switching everyone around, plus I got grumpy and felt off my game.

also, I have my students free-write for 5 minutes before they start peer review on peer review days about the positives and negatives of their papers and then specific things they'd like their reviewers to pay attention to.

hmmm. I may consider this. I do have a tight peer review schedule though.

After that, maybe an additional concrete suggestion or two is all that they can meaningfully handle.

Hmm. Overload. Can you do a check-off-the-topic-sentences type peer review and still get at the idea of global revision?

But the biggest thing is to just them get comfortable enough with each other to say things like "I'm confused--what are you trying to do here?" to each other.

They are *very* comfortable with each other --- they were telling each other all sorts of interesting things in the last groups that I, had I actually heard, would have probably had to report to the police or campus authorities. I need to put the fear of Sisyphus in them.

Sisyphus said...

Oh, p.s. I also give them points for peer editing as long as they "do a good job" - whatever that means - and have a fully finished rough draft to exchange. Usually that means that the people without drafts get zero points. So just showing up with a draft is like points for free. But not showing up with the draft is really a downer on the grade.

Yup, I do that. Don't have the length of time in class or the length of the papers to do the other bit.

tudents bring a draft of their paper, with enough copies for everyone in their group (usually four students per group, sometimes five) and a copy for the instructor. They take the papers home, read them, and comment on them using a worksheet.

I don't get it; that's a pretty standard peer review/conference format. I'm only doing the conference part one time because the department mandates at least one individual or group student conference, and I don't get paid enough to run through that with 75 comp students multiple times. I'm already dreading the one required time.

From the looks of the Harris book on Amazon preview, my students are not at that capability. I'm really reaching over their heads in making them respond to and quote texts, which is not at all the dept. norm --- the lecturers I talk to are doing personal essays until the end of the semester, no quotes.

The first go is random.

I totally should have done this --- I made groups based on their response paper quality so far, and then had a total change-up when lots of them didn't show or came late and I ended up switching everyone around, plus I got grumpy and felt off my game.

also, I have my students free-write for 5 minutes before they start peer review on peer review days about the positives and negatives of their papers and then specific things they'd like their reviewers to pay attention to.

hmmm. I may consider this. I do have a tight peer review schedule though.

After that, maybe an additional concrete suggestion or two is all that they can meaningfully handle.

Hmm. Overload. Can you do a check-off-the-topic-sentences type peer review and still get at the idea of global revision?

But the biggest thing is to just them get comfortable enough with each other to say things like "I'm confused--what are you trying to do here?" to each other.

They are *very* comfortable with each other --- they were telling each other all sorts of interesting things in the last groups that I, had I actually heard, would have probably had to report to the police or campus authorities. I need to put the fear of Sisyphus in them.

midprof said...

I have been unconvinced of the value of peer review in the past. This term, though, I developed a short assignment (3-4 pp) which students handed in, bringing 2 copies. On the day they handed it in, I had them hand it to each other, and each person had two readers. It was a time suck, but since the papers were short, it was doable. I found the trick was giving the students super-specific prompts in the peer review. They wrote on a handout rather than on the paper, answering such things as : 1) does the writer identify the most important arguments of x, and if so, where; 2) has the writer characterized the type of journal from which y came, if so, where, etc. I urged them away from being copy-editors; we were trying to mirror the peer review process they were learning about in "real" scholarship. Having such specific prompts enabled me to check whether writers then actually responses to the peer review properly (gee, peer reviewer a told you that your citation format was incorrect, and yet, it's still there!) It also enabled me to form a rubric (first time) to grade the papers. I gave out the toughest grades I've given in a while, and am eager to give it a second try.