The Need to Evaluate Question Writers’ Views and the Costs of Constructed-Response Versus Selected-Response Calculations Questions

To the Editor. Sheaffer and Addo wrote an interesting account of their experience with constructed-response vs selected-response calculations questions.¹ Even though the results do contribute to a growing evidence base in this field, it is perhaps not surprising that students will prefer and perform better on those question types with which they are more familiar.

It would have been interesting to hear the views of authors who have to write constructed-response and selected-response questions. Perhaps it is too much to ask students who have to answer questions to sympathize with those who write them; however, writing questions of either type is not straightforward. Question writers must come up with assessments that are fair, valid, and reliable, and that have a positive effect on students’ learning behaviors.²

Writing questions is a skill that must be learned like any other skill, and such skills are not necessarily transferable from one question type to another. It would have been interesting to know what question-writers thought of the new question format and whether they thought that the new format made their job of producing good questions easier or harder. It would also have been interesting to know whether their views changed over time as they developed experience with the new question format.

It would also have been interesting to find out the comparative costs of creating constructed-response vs selected-response calculation questions. Much of the costs will inevitably relate to question writers’ time or pay, and training in question writing. Costs will need to be allocated to blue printing and standard setting. There is also the cost of administrators to manage the question-writing process. Finally, there are the costs of hardware and software. It is tempting to believe that some of these costs would be higher or lower depending on the question type. However, in the absence of evidence pointing one way or the other, this amounts only to speculation. Comparing the costs and other utility outcomes of different question types is not an area where much work has been done – hopefully, this will make it an attractive domain for educational researchers.

• © 2013 American Association of Colleges of Pharmacy