Peer Review – some lessons learned & some friendly advice

Inspired by events and discussions at the recent Y1 Feedback Symposium, I’ve been mulling over how I can improve the peer review process in an online staff development course that I teach. During his presentation, assessment researcher Professor David Nicol made the point that students seem to learn more in conducting a feedback review than actually being the receiving party. The Nicol, Thomson, and Breslin (2014) research on peer review makes convincing reading about the evaluative and cognitive benefits of the review process, and  I’ve decided to share that paper with my own students to explain why we’re using this approach.

This got me thinking about how I’ve used peer review (actually, I called it peer critique, which is probably not ideal) in the past. While not claiming to have vast experience with the approach, I have used it with a class of online students who were asked to give each other feedback on proposed strategies. Some unexpected issues came to light:

  1. Some peer feedback comments were interpreted as being undiplomatic and irrelevant
  2. Providing what I thought were helpful ‘prompts’ for questions/comments ended up being regurgitated directly, in a small number of cases
  3. While some students loved it (particularly those working in similar disciplines), some participants were unconvinced (as they felt they did not receive the type or quality of feedback they would expect)
  4. Because students were free to discuss using any communication mode of their choice, several offline conversations were not visible to other students or me
  5. Some students did not engage at all with the process and some left significant elements out (eg did not explain what aspects of feedback they planned to incorporate or omit)

There appears to be a fine line between providing appropriate scaffolding and micro-managing the process for students: you want to give them enough information to know how to get started with a peer review but not so much that it becomes a simplistic or somewhat pointless exercise. At the symposium, Prof. Nicol made the point that students should ideally generate their own criteria for quality when conducting peer review. While I might see this working in a small group face-to-face setting, I am not sure it would be as successful in an online context where silence (the equivalent of the blank page) is simply easier to ignore. So I suspect the online learner requires the provision of at least some guidelines as a starting point for the discussion and you can find helpful advice on peer feedback forms from the University of Hawaii, Manoa. Of course, to a large extent it depends on the course design but in the online context I think I’d prefer to play it safe by providing at least some suggested criteria or guidelines at the start.

So trying to tie all this neatly together, here are some ideas for the next time I use a peer review approach in online teaching:

  • Record a screencast or video that captures my thoughts ‘thinking out loud’ as I read and annotate a sample draft (this might help to model suggested feedback and tone, addressing points 1,2 and 3)
  • Set up a central location (such as a discussion forum) with a designated thread for each pair to respond to online (this might assist with points 3 and 4 by increasing visibility into the process and enabling greater access to other classmates’ reviews)
  • Create an infographic that provides a recommended pathway for the assignment, from beginning to end (this might clarify the multiple steps involved, addressing point 5)

As luck would have it, my 14-year-old daughter has been doing a peer review for her English class in recent times and has been subject to intense questioning about it by her mother. I should add that she is completely unimpressed by it as a learning device (sigh) and reckons that it is too hard to “be honest” with one’s friends. I know it is the job of a 14-year-old to be unimpressed by everything but she might have a point about that – could anonymity help or hinder the peer review process?

If anyone else has comments on proposed or previous experiences of online peer review, I’d love to hear your thoughts. Speak up, my friends.

References

Nicol, D., Thomson, A. & Breslin, C. 2014, “Rethinking feedback practices in higher education: a peer review perspective”, Assessment & Evaluation in Higher Education, vol. 39, no. 1, pp. 102-122.

 

Advertisements

3 thoughts on “Peer Review – some lessons learned & some friendly advice

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s