147x Filetype PDF File size 0.25 MB Source: melbourne-cshe.unimelb.edu.au
MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES MULTIPLE CHOICE QUESTIONS: AN INTRODUCTORY GUIDE Elisa Bone and Mike Prosser Multiple-choice questions (MCQs) continue to PART 1: Role and Function of MCQs be relied upon for the efficient assessment of MCQs can be used to play a wide variety of roles students’ learning in higher education. However, in our teaching and learning. They can be used in the use of MCQs has not been without criticism. lectures and online activities to provide feedback Poor design of MCQs, testing only lower-level to students. Their use in clickers and polls within 1 learning outcomes such as information recall, large lectures classes and their periodic use in and encouraging students to focus on these rote online learning activities can be key to providing and reproductive approaches to learning are immediate formative feedback to students and among the criticisms. Each of these criticisms staff. They can efficiently cover a wide range of can be mitigated with close attention to the topics, and be reliably and objectively marked function, purpose and design of MCQs in the or scored without the need for a marking rubric. curriculum. This Guide focusses on addressing With the development of a large bank of items and these issues and helping to make the use of MCQs appropriate software, they can be used to provide in assessment of learning more valid, reliable and each student with a different test, supporting their productive. use of assessment of learning online. The first part of the Guide addresses issues of role But, among the criticisms of the use of MCQs, and function, where we will discuss the outcomes three stand out: of research and scholarship into these aspects in relation to assessment for student learning. 1. They tend to focus on the assessment of low- level knowledge and encourage students to The second part of the Guide describes the major adopt rote learning approaches. forms and structures of MCQs and discusses their 2. The design of items testing high-level advantages and disadvantages, with a focus on knowledge is time consuming and requires the design of MCQs for the assessment of student expertise and experience. learning. 3. Their use distorts students approaches to We finish with a short section on Do’s and Don’ts study – students may perceive that all they of using MCQs and throughout this guide, our need to do is to memorise enough material major focus is on the valid and reliable assessment and they will pass. of student learning. 1 For further information on learning outcomes and their alignment to the curriculum, please, see the MCSHE resource ‘Writing Learning Outcomes: A Practical Guide for Academics’ by. Available at: https://melbourne-cshe.unimelb.edu.au/resources/categories/teaching-and-learning/curriculum-design MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, MAY 2020 1 MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES A great deal of systematic and anecdotal A. Incorporating MCQs, including those testing evidence supports the assumption that MCQs higher-level learning, into lecture, tutorial tend to assess low-level knowledge and there and online programs and clearly articulating are several reasons why this is so. Among expected learning outcomes. This will help them is that the design of items to test high students understand what is being assessed. level knowledge and understanding is difficult B. Using MCQs to only test low-level knowledge, and time consuming. The design of items that whilst using short open-ended questions to are valid and reliable indicators of high-level test high-level knowledge. knowledge and understanding, with well-designed C. Using MCQs to test both low-level and distractors, requires expertise and experience. It is high-level knowledge within formative comparatively much easier to design items to test assessments, with expected learning low-level knowledge and understanding. Indeed, outcomes clearly articulated and directed it can be argued that the efficiency in marking feedback provided. items measuring high-level knowledge and understanding may not compensate for the time Suggestion A gives students a mix of low-level and needed to develop valid and reliable measures high-level MCQs, asking them to answer the items, of such knowledge and understanding. We will then asking them to classify the items as testing address ways of mitigating these risks later in our low-level or high-level knowledge and finally list of Do’s and Don’ts. discussing their responses contributes to both The effect on students’ approaches to study also assessment of, and for, learning. Using this strategy 2,3 students experience the items and are assisted to needs consideration . Rote and reproductive reflect on them, building their understanding of approaches to learning are common. The the expected levels of achievement in the subject. evidence is that students perceive MCQs to test Incorporating these interactive sessions low-level knowledge and understanding. The very structure of MCQs tends to evoke such a Suggestion B incorporates the issue of difficulty in perception. Consequently, students tend to adopt design of high-level items and students’ perceptions ‘surface’ approaches to studying for MCQ tests, that they test low-level knowledge. In this approach, perceiving that if they can remember ‘enough relatively easily designed MCQs that are used to test stuff’ then they can do well. It is difficult to change low-level knowledge are supplemented with short this perception. Consequently, when well- open-ended questions to test high-level knowledge designed items measuring high-level knowledge and understanding. This approach mitigates and understanding are used, students often do against the risk of poorly designed high-level items poorly. Suggested ways for mitigating this risk are: and is consistent with students’ perceptions of MCQs and open-ended questions. Suggestion C provides a compromised approach and can be useful in cases where, for example, in-depth discussion is not feasible, or assessments are largely online. Feedback can be tailored to discuss both the correct and incorrect alternatives, allowing students to reflect on their reasoning. Feedforward processes can also be incorporated to allow comments and reflections from students, which may in turn be utilised to inform the design of future assessment tasks. 2 Scouller, K. M., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19(3): 267–279. 3 Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35: 453–472. MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 2 MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES PART II: Writing MCQs Ensure that the stem and alternatives make An MCQ comprises the stem, a question or sense on their own without reference to problem that leads into the list of possible additional materials and use terms and solutions, or alternatives, comprising both the symbols that are familiar to students. This will key (the correct answer) and several incorrect focus students’ attention on answering the distractors. MCQ rather than interpreting its construction, reducing their cognitive load. Similarly, use The stem may take the form of a question or simple sentence structures, avoiding non- statement and may also refer to external or standard lingo and jargon, complex grammar supplementary material, such as a figure or and lengthy unnecessary wording. diagram, which students are asked to interpret Avoid using negative language within the in selecting the best alternative. Similarly, stem, including double negatives within alternatives can be in the form of single terms, both the stem and alternatives, unless the statements, figures or diagrams. learning objectives call for such language. There are several key considerations that need Using negative language can both increase a to be applied when writing MCQ items. Taken students’ cognitive load, by requiring them to together, these aim to ensure that MCQ questions examine the question more carefully to detect are reliable – consistently measuring a learning this language, and decrease the question’s outcome, are valid – testing student learning at the validity, since students who fail to detect the appropriate level, and are focused, with students negative language will be less likely to answer spending most of their time considering their the question correctly. If using negative response to the question, rather than on making language, ensure it is flagged or highlighted, sense of extraneous or irrelevant material. So, for example in bold text or capitalisation. when writing an MCQ question, one should: Brame (2013) and Chiavaroli (2017) discuss these aspects in more detail. Make sure all alternatives are plausible and In the following section, we present several relate in some way to the topic or subject examples of MCQ items, of different structural matter. This will increase the question’s forms, that aim to assess both low-level learning reliability – increasing the chances that outcomes such as recall and comprehension, and students who choose the correct answer higher-level learning outcomes such as analysis, meet the learning objective. evaluation and synthesis. Write alternatives that are as similar as possible in grammar, length, language and form. This will reduce the chances that students may detect hidden clues as to which alternative is the correct one, assisting to increase the question’s validity. MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 3 MELBOURNE CSHE TEACHING AND LEARNING SHORT GUIDE SERIES Examples of common MCQ types Presented in this section are examples of MCQs extracted from popular textbook question banks, from practice or from the literature. For each question, we present a brief evaluation and point out strengths and weaknesses in the question design. In the appendix, we discuss a broader range of MCQs in more detail. The correct answer for each question is denoted by an asterisk *. A. MCQs to test lower-level learning outcomes (recall, comprehension) 1. Choose the best answer ✓ Stem is posed as a question and relates to From where do most fungi obtain nutrients? general topic. ✓ Each alternative answer is plausible. A. Nonliving organic matter* ✓ Alternatives are about the same length. Avoid including concepts that require B. Living plants clarification or imply restrictions or absolutes, C. Living animals e.g. ‘only’, ‘most’, ‘always’. D. Photosynthesis 2. Fill in the blank (#1) ✓ Each alternative answer is a single item. Autotrophs use as an energy source ✓ All alternative choices should be familiar terms to to drive photosynthesis. the students. Alternatives are in different forms – both words A. Sunlight* and molecular formulae. Use consistent formats. The question stem does not stand alone – B. Hydrogen ions students will need to evaluate sentence for each alternative. C. O2 Avoid this format in timed exams and quizzes. D. CO2 3. Fill in the blank (#2) ✓ Each alternative answer is a single item. Light-dependent reactions in plants proceed in the ✓ Each alternative answer is plausible. ✓ All alternative choices use terms familiar to the students. A. Thylakoid membrane* The question stem is a partial sentence. Writing the stem as a full question is preferable. B. Plasma membrane C. Stroma D. Cytoplasm 4. Complete the statement ✓ Each alternative answer is plausible. A mushroom is ✓ Alternatives test students’ understanding of concepts. A. The digestive organ of a club fungus Avoid including concepts that require clarification or imply limitation/restriction/ B. The only part of the fungal body made of hyphae absolutes, e.g. ‘only’, ‘most’, always. C. A reproductive structure that releases sexual spores* D. The only diploid phase in the club fungus life cycle MELBOURNE CENTRE FOR THE STUDY OF HIGHER EDUCATION, JULY 2020 4
no reviews yet
Please Login to review.