Filetype PDF | Posted on 02 Feb 2023 | 2 years ago
Authentication
digital resources
149x Filetype PDF File size 0.25 MB Source: melbourne-cshe.unimelb.edu.au
MELBOURNE CSHE
TEACHING AND LEARNING
SHORT GUIDE SERIES
MULTIPLE CHOICE QUESTIONS:
AN INTRODUCTORY GUIDE
Elisa Bone and Mike Prosser
Multiple-choice questions (MCQs) continue to PART 1: Role and Function of MCQs
be relied upon for the efficient assessment of MCQs can be used to play a wide variety of roles
students’ learning in higher education. However, in our teaching and learning. They can be used in
the use of MCQs has not been without criticism. lectures and online activities to provide feedback
Poor design of MCQs, testing only lower-level to students. Their use in clickers and polls within
1
learning outcomes such as information recall, large lectures classes and their periodic use in
and encouraging students to focus on these rote online learning activities can be key to providing
and reproductive approaches to learning are immediate formative feedback to students and
among the criticisms. Each of these criticisms staff. They can efficiently cover a wide range of
can be mitigated with close attention to the topics, and be reliably and objectively marked
function, purpose and design of MCQs in the or scored without the need for a marking rubric.
curriculum. This Guide focusses on addressing With the development of a large bank of items and
these issues and helping to make the use of MCQs appropriate software, they can be used to provide
in assessment of learning more valid, reliable and each student with a different test, supporting their
productive. use of assessment of learning online.
The first part of the Guide addresses issues of role But, among the criticisms of the use of MCQs,
and function, where we will discuss the outcomes three stand out:
of research and scholarship into these aspects in
relation to assessment for student learning. 1. They tend to focus on the assessment of low-
level knowledge and encourage students to
The second part of the Guide describes the major adopt rote learning approaches.
forms and structures of MCQs and discusses their 2. The design of items testing high-level
advantages and disadvantages, with a focus on knowledge is time consuming and requires
the design of MCQs for the assessment of student expertise and experience.
learning.
3. Their use distorts students approaches to
We finish with a short section on Do’s and Don’ts study – students may perceive that all they
of using MCQs and throughout this guide, our need to do is to memorise enough material
major focus is on the valid and reliable assessment and they will pass.
of student learning.
1 For further information on learning outcomes and their alignment to the curriculum, please, see the MCSHE resource ‘Writing Learning Outcomes: A
Practical Guide for Academics’ by. Available at: https://melbourne-cshe.unimelb.edu.au/resources/categories/teaching-and-learning/curriculum-design
MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, MAY 2020 1
MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES
A great deal of systematic and anecdotal A. Incorporating MCQs, including those testing
evidence supports the assumption that MCQs higher-level learning, into lecture, tutorial
tend to assess low-level knowledge and there and online programs and clearly articulating
are several reasons why this is so. Among expected learning outcomes. This will help
them is that the design of items to test high students understand what is being assessed.
level knowledge and understanding is difficult B. Using MCQs to only test low-level knowledge,
and time consuming. The design of items that whilst using short open-ended questions to
are valid and reliable indicators of high-level test high-level knowledge.
knowledge and understanding, with well-designed C. Using MCQs to test both low-level and
distractors, requires expertise and experience. It is high-level knowledge within formative
comparatively much easier to design items to test assessments, with expected learning
low-level knowledge and understanding. Indeed, outcomes clearly articulated and directed
it can be argued that the efficiency in marking feedback provided.
items measuring high-level knowledge and
understanding may not compensate for the time Suggestion A gives students a mix of low-level and
needed to develop valid and reliable measures high-level MCQs, asking them to answer the items,
of such knowledge and understanding. We will then asking them to classify the items as testing
address ways of mitigating these risks later in our low-level or high-level knowledge and finally
list of Do’s and Don’ts. discussing their responses contributes to both
The effect on students’ approaches to study also assessment of, and for, learning. Using this strategy
2,3 students experience the items and are assisted to
needs consideration . Rote and reproductive reflect on them, building their understanding of
approaches to learning are common. The the expected levels of achievement in the subject.
evidence is that students perceive MCQs to test Incorporating these interactive sessions
low-level knowledge and understanding. The
very structure of MCQs tends to evoke such a Suggestion B incorporates the issue of difficulty in
perception. Consequently, students tend to adopt design of high-level items and students’ perceptions
‘surface’ approaches to studying for MCQ tests, that they test low-level knowledge. In this approach,
perceiving that if they can remember ‘enough relatively easily designed MCQs that are used to test
stuff’ then they can do well. It is difficult to change low-level knowledge are supplemented with short
this perception. Consequently, when well- open-ended questions to test high-level knowledge
designed items measuring high-level knowledge and understanding. This approach mitigates
and understanding are used, students often do against the risk of poorly designed high-level items
poorly. Suggested ways for mitigating this risk are: and is consistent with students’ perceptions of
MCQs and open-ended questions.
Suggestion C provides a compromised approach
and can be useful in cases where, for example,
in-depth discussion is not feasible, or assessments
are largely online. Feedback can be tailored to
discuss both the correct and incorrect alternatives,
allowing students to reflect on their reasoning.
Feedforward processes can also be incorporated
to allow comments and reflections from students,
which may in turn be utilised to inform the design
of future assessment tasks.
2 Scouller, K. M., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19(3): 267–279.
3 Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment
essay. Higher Education, 35: 453–472.
MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 2
MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES
PART II: Writing MCQs Ensure that the stem and alternatives make
An MCQ comprises the stem, a question or sense on their own without reference to
problem that leads into the list of possible additional materials and use terms and
solutions, or alternatives, comprising both the symbols that are familiar to students. This will
key (the correct answer) and several incorrect focus students’ attention on answering the
distractors. MCQ rather than interpreting its construction,
reducing their cognitive load. Similarly, use
The stem may take the form of a question or simple sentence structures, avoiding non-
statement and may also refer to external or standard lingo and jargon, complex grammar
supplementary material, such as a figure or and lengthy unnecessary wording.
diagram, which students are asked to interpret Avoid using negative language within the
in selecting the best alternative. Similarly, stem, including double negatives within
alternatives can be in the form of single terms, both the stem and alternatives, unless the
statements, figures or diagrams. learning objectives call for such language.
There are several key considerations that need Using negative language can both increase a
to be applied when writing MCQ items. Taken students’ cognitive load, by requiring them to
together, these aim to ensure that MCQ questions examine the question more carefully to detect
are reliable – consistently measuring a learning this language, and decrease the question’s
outcome, are valid – testing student learning at the validity, since students who fail to detect the
appropriate level, and are focused, with students negative language will be less likely to answer
spending most of their time considering their the question correctly. If using negative
response to the question, rather than on making language, ensure it is flagged or highlighted,
sense of extraneous or irrelevant material. So, for example in bold text or capitalisation.
when writing an MCQ question, one should: Brame (2013) and Chiavaroli (2017) discuss
these aspects in more detail.
Make sure all alternatives are plausible and In the following section, we present several
relate in some way to the topic or subject examples of MCQ items, of different structural
matter. This will increase the question’s forms, that aim to assess both low-level learning
reliability – increasing the chances that outcomes such as recall and comprehension, and
students who choose the correct answer higher-level learning outcomes such as analysis,
meet the learning objective. evaluation and synthesis.
Write alternatives that are as similar as
possible in grammar, length, language and
form. This will reduce the chances that
students may detect hidden clues as to which
alternative is the correct one, assisting to
increase the question’s validity.
MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 3
MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES
Examples of common MCQ types
Presented in this section are examples of MCQs extracted from popular textbook question banks, from
practice or from the literature. For each question, we present a brief evaluation and point out strengths
and weaknesses in the question design. In the appendix, we discuss a broader range of MCQs in more
detail. The correct answer for each question is denoted by an asterisk *.
A. MCQs to test lower-level learning outcomes (recall, comprehension)
1. Choose the best answer
✓ Stem is posed as a question and relates to
From where do most fungi obtain nutrients? general topic.
✓ Each alternative answer is plausible.
A. Nonliving organic matter* ✓ Alternatives are about the same length.
Avoid including concepts that require
B. Living plants clarification or imply restrictions or absolutes,
C. Living animals e.g. ‘only’, ‘most’, ‘always’.
D. Photosynthesis
2. Fill in the blank (#1)
✓ Each alternative answer is a single item.
Autotrophs use as an energy source ✓ All alternative choices should be familiar terms to
to drive photosynthesis. the students.
Alternatives are in different forms – both words
A. Sunlight* and molecular formulae. Use consistent formats.
The question stem does not stand alone –
B. Hydrogen ions students will need to evaluate sentence for each
alternative.
C. O2 Avoid this format in timed exams and quizzes.
D. CO2
3. Fill in the blank (#2)
✓ Each alternative answer is a single item.
Light-dependent reactions in plants proceed in the ✓ Each alternative answer is plausible.
✓ All alternative choices use terms familiar to the
students.
A. Thylakoid membrane* The question stem is a partial sentence.
Writing the stem as a full question is preferable.
B. Plasma membrane
C. Stroma
D. Cytoplasm
4. Complete the statement
✓ Each alternative answer is plausible.
A mushroom is ✓ Alternatives test students’ understanding of
concepts.
A. The digestive organ of a club fungus Avoid including concepts that require
clarification or imply limitation/restriction/
B. The only part of the fungal body made of hyphae absolutes, e.g. ‘only’, ‘most’, always.
C. A reproductive structure that releases sexual spores*
D. The only diploid phase in the club fungus life cycle
MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 4
no reviews yet
Please Login to review.
