Students writing assessments?

Picture the scene: it's that time of year again when your exam questions are due (overdue, perhaps?) You settle down to the task at hand and after several hours, many cups of coffee and a number of blind alleys of unworkable yet elegant questions. you declare it finished. A work of beauty: easy to get going with at the start, yet with a suitable steepening of difficulty as you progress and a fearsome sting in the tail. Anyone who sets examination or assessment questions will undoubtedly recognise this feeling: the satisfaction of having crafted something plausible, contextualised, appropriate. It's a cognitively demanding task to set good assessment questions.

Why not ask the students? 

During their undergraduate degree, students are expected to solve hundreds (if not thousands) of practice problems, examples, exercises and assessment questions. No doubt many of these do act as demanding hurdles for students, but - in the spirit of variety - why shouldn't we task them with the very authentic activity of writing questions, providing plausible answers (right and wrong), together with the associated explanations and solutions? 

Perhaps it is because many higher education courses in the sciences are still obsessed with covering content, even if it means that in doing so you don't uncover much lasting understanding. A recent research study¹ classified nearly 10 000 assessment questions used in biology at tertiary level from 50 instructors in the US and found nearly all of them (93%) tested just recall or basic understanding. 

In our first year courses at Edinburgh, we think we can offer students more than this. As part of their coursework, we've been asking students to become assessment creators, writing their own multiple choice questions, with explanations and solutions. These are submitted to an online repository, where they and their peers can answer, discuss, comment on and rate the body of questions, forming an active and vibrant online community of practice. 

We've been using a freely-available online system called PeerWise,² developed by the Computer Science department at the University of Auckland in New Zealand. Used extensively across the world, in both secondary and tertiary education, PeerWise is straightforward to use, has a clean interface and design, and embodies much of the social functionality that now forms the backbone of online interaction. Students can follow particular question authors, and be informed when they contribute new material. Updates and messages are delivered to a user at each login, telling them how other users have been interacting with their content. There's an in-built scoring system, and a selection of badges to collect to incentivise and motivate contributors, and to encourage participation and engagement. And, of course, it's easy to import chemical structure diagrams, reaction mechanisms or mathematics into questions and their answers. 

Simply offering students the system to use without any training - the 'build it and they will come' approach - is not enough. In piloting PeerWise, we've found that there are a few key implementation details that are necessary for high quality student engagement. We found it essential to provide a route into the difficult task of creating questions from scratch. Where do you start? How do you know what you don't know? We devised scaffolding activities³ to be done in groups in class ahead of the start of the assessment task. This turned out to be crucial in setting a high standard for students to aspire to. 

Key to success 

To ensure good participation from all of the class, there was an incentive, in the form of a small amount of assessed course credit. This was only a few percent of the total course mark, but it ensured a high take up of the task. 

The single biggest surprise to us when implementing this activity was the high quality of nearly all the questions, with the very best ones being outstandingly good. Colleagues asked us if we were not concerned that students would plagiarise end-of-chapter problems from a textbook? On the whole, no: many of the problems students submitted were far better than end of chapter problems. 

We are currently conducting a JISC-funded project⁴ investigating the rollout of this system to courses in physics, chemistry and biology at Edinburgh. We have also successfully implemented it with a class of 13 year olds, achieving excellent results. Our research is showing a correlation between students (all ability ranges) who make greater use of the system and higher end-of-course exam marks.⁵ 

This might sound all too much for some conservative types, but perhaps, in an increasingly collaborative and connected world, should we not be doing more to develop students as creators as well as consumers of knowledge?

Simon Bates is Dean of Learning and Teaching in the College of Science and Engineering, and Professor of Physics Education at the University of Edinburgh

Related Links

 EiC on Twitter 

Follow us on Twitter @RSC_EiC

 EiC Facebook Page 

Become a fan of Education in Chemistry on our Facebook page

 EiC on LinkedIn 

Join the Education in Chemistry group on LinkedIn

Contact and Further Information

EiC Editorial Office 
Royal Society of Chemistry, Thomas Graham House, Science Park, Milton Rd, Cambridge CB4 0WF
Tel: +44 (0)1223 420066 
Email: EiC Editorial Office