Does question bank design affect student performance?

We know it is important to provide students with repeated opportunities to practise solving questions. In a new study, researchers examined how the presentation and format of practice questions influences students’ problem-solving performance. The study revealed that mixed problem sets are better than questions arranged by topic.

There are two types of question practice teachers can give their students. The first, blocked practice, involves solving multiple problems of the same type, or about the same concept, before moving on to another. Practice worksheets and end-of-chapter questions in textbooks are often blocked practice type questions. They hone students’ algorithmic problem-solving skills, but at the expense of their conceptual understanding of the topic. The second, interleaved practice, shuffles between different types of questions in one session. It is more difficult, because students must identify the type of question being asked, or the concept it relates to, in addition to answering. Shuffled questions are thought to help with long-term learning and are similar to the questioning format in students’ exams.

The researchers investigated the effects of these two kinds of practice. They recruited 79 university students from general chemistry classes. They gave one group assignments with mixed questions and a control group assignments with questions organised into topics or chapters. They compared the groups’ performances through one pre-test and post-test after each of three problem-solving sessions.

Teaching tips

  • Source or compile more mixed practice question banks, and avoid solely using topic-specific question sets from textbooks.
  • To increase the difficulty and benefits of mixed problem question sets, do not include details identifying the relevant topics or chapters.
  • Blocked practice still has a place. Certain topics, particularly those to do with numerical problem-solving, may require extensive blocked practice before students can engage with the benefits of interleaved practice.

Probing the problems

Rather than looking at overall scores, the researchers used a more detailed analysis. They broke each problem into the sub-problems, or steps, required to answer the problem. They then categorised students’ answers to the steps as successful, neutral or unsuccessful. Subcategories provided more insight into the students’ work. For example, the neutral category contained the subcategories ‘not required’, ‘did not know to do’ and ‘did something else’.

The study revealed that students in the interleaved-practice group increased their problem-solving success more than those in the blocked-practice group. Significantly, the achievement gap between the experimental and control groups widened as the study progressed. Following interleaved practice, students’ neutral codes decreased by about 70%, unsuccessful codes decreased by about 40%, and the successful codes increased by about 52%. Even if students were not able to complete the entire problem, they still improved at individual steps.

Importantly, even though A-, B- and C-grade students showed different levels of improvement, they all benefited from interleaved practice. Perhaps unexpectedly, B- and C-grade students improved the most. This might relate to their poorer conceptual understanding of topics or assessment literacy beforehand, which interleaved practice helps to develop.

Fraser Scott

Reference

O. Gulacar et al, Chem. Educ. Res. Pract., 2022, DOI: 10.1039/D1RP00334H