Assessment Questions by Kalai Selvi Arivalagan (free e reader .TXT) 📕

Guidelines

Skill in Techniques of Item Writing

 

Before we start on discussing the challenges in item writing and the ways to overcome the hurdle of writing better, we must know the difference between literal, inference and critical questions. Literal questions ask the students to find specific details stated within the text. Inference and critical questions require the students to stop and think about the text.

 

What is a Multiple Choice Item?

 

The multiple choice item (MCQ) consists of two distinct parts:

 

1. The first part that contains task or problem is called stem of the item. The stem of the item may be presented either as a question or as an incomplete statement. The form makes no difference as long as it presents a clear and a specific problem to the examinee.

 

2. Second part presents a series of options or alternatives. Each option represents possible answer to the question. In a standard form one option is the correct or the best answer called the keyed response and the others are misleads or foils called distracters.

 

The number of options used differs from one test to the other. An item must have at least

three answer choices to be classified as a multiple choice item. The typical pattern is to have

four or five choices to reduce the probability of guessing the answer. A good item should

have all the presented options look like probable answers at least to those examinees who

do not know the answer.

 

Parts of a multiple choice question (Bull & Mckenna, 2002)

A traditional multiple choice question (or item) is one in which a student chooses one answer from a number of choices supplied. A multiple choice question consists of a:

stem - the text of the question

options - the choices provided after the stem (these include the key and the distracters)

the key - the correct answer in the list of options

distracters - the incorrect answers in the list of options

Terminology : Multiple Choice Questions

 

1.Stem: presents the problem

2. Keyed Response: correct or best answer

3.Distracters: appear to be reasonable answers to the examinee who does not know the content

4. Options: include the distracters and the keyed response.

 

Common Errors in Writing Multiple-Choice Items:

 

Item stems that are too lengthy or imprecise.

Poor: Which one of the following is something that can be found in a short story that is also found in this personal narrative?

Corrected: Which of these short story elements is present in this personal narrative?

Item stems that are choppy or abstract. Test takers need to have a clear picture of their task before they begin reading the answer choices.

Poor: Reading this chart - A) can tell someone the growth of four stocks over a four-year period

Corrected: This chart shows the growth of – A) four stocks over a four-year period.

Item options that lack balance. This problem can be seen in several ways, including “stacked” options, options that are dramatically different in “flavor,” and one option that is obvious in some way.

Poor: Animal tracks can be best found in

mud or snow (stacked option – it contains two elements)

grass

water

the library (option is dramatically different in “flavor”)

Corrected: Animal tracks can be best found in a) mud b) grass c) water d) sand

Item stems and their options that needlessly repeat words.

Poor: Penguins cannot fly because

their wings are too small

their wings are too flat

their wings are too curved

their wings are too heavy

Corrected: Penguins cannot fly because their wings are too

small

flat

curved

heavy

A stimulus, item stem, or an option containing information that clues the correct answer of the item itself or of another item in the test.

Items that measure more than one standard.

Example: Holly sitted next to the band leader.

 

How should this sentence be written?

Holly sat next to the band leader.

Holly sit next to the band leader.

Holly sitting next to the band leader.

holly sat nexst to the band leader.

In this case, the item was intended to measure the standard “Use correct verb formation.” In this item, options a, b and c provide various forms of the verb. Option D tests capitalization and spelling (which are measured under separate standards). Therefore, this item is assessing a student’s ability to do more than merely choosing the correct verb in the sentence. The option d need to be revised (only the verb contains the error) to match the standard.

 

Guidelines for Response Options:

 

No response option (correct answer or distracter) should be significantly different from the other three options (in length, complexity, structure). However, pairs of options may reflect minor differences (two long and two short options).

The response options should relate to the stem in the same way both grammatically and conceptually.

No response option (correct answer or distracter) should eliminate another response action. Thus two distracters should not be opposites of each other.

No response option (correct answer or distracter) should deny the truth of the stem.

Response options should not give clues to students. Avoid the use of templates (never, always, all). Words used in the stem should not be repeated in the response options (correct answer or distracter).

Avoid repeating words or phrases in the options that could be placed in the stem.

The distracters should be plausible and reasonable in terms of the stimulus, the item stem, and the student’s level of knowledge.

Distracters may include a) common misconceptions or misunderstandings or b) misreading of the stimulus. For some tests, it is helpful for item writers to include the rationales (reasons) for the distracters.

Do not use distracters that have nearly the same meaning even though they are worded differently. (Julian was happy to go. Julian was looking forward to the trip.)

 

Submission Checklist:

 

When you are ready to submit your materials, please be sure you have completed the following:

 

Read and understand the project’s guiding documents (example: test and item specifications)

Follow the guidelines in the Item Writer’s Guide.

Create original, interesting, grade-appropriate items that measure the standards.

Use the review checklists for each item you are submitting to ensure it is of the highest quality.

Complete the template provided for each item and follow submission guidelines.

Verify with source materials for factual information.

 

14 Rules for Writing Multiple-choice questions

 

1. Use Plausible Distracters (wrong-response options)

• Only list plausible distracters, even if the number of options per question changes.

• Write the options so they are homogeneous in content.

• Use answers given in previous open-ended exams to provide realistic distracters.

 

2. Use a Question Format

• Experts encourage multiple-choice items to be prepared as questions (rather than incomplete statements)

 

 

Incomplete Statement Format:

 

The capital of California is in ____________. Less Effective

 

Direct Question Format:

 

In which of the following cities is the capital of California? More Effective

 

3. Emphasize Higher-Level Thinking

 

• Use memory-plus application questions. These questions require students to recall principles,

rules or facts in a real life context.

• The key to preparing memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation.

• Seek support from others who have experience writing higher-level thinking multiple-choice questions.

 

Memory Only Example

Which description best characterizes whole foods?

a. orange juice

b toast                                           Less Effective

c. bran cereal

d. grapefruit

 

Memory-Plus Application Example

 

Sally’s breakfast this morning included one glass of orange juice (from concentrate), one slice of toast, a small bowl of bran cereal and a grapefruit. What “whole food” did Sally eat for breakfast?

a. orange juice

b. toast                                                Effective

c. bran cereal

d. grapefruit

 

Memory-Plus Application Example

                                                                            Effective

Which one of the following best illustrates the law of diminishing returns?
a. The demand for a farm product increased faster than the supply of the product.

b. The population of a country increased faster than the means of subsistence.

c. A machine decreased in utility as its parts became worn.

d. A factory doubled its labor force and increased production by 50 percent

 

Ability to Interpret Cause-and-Effect Relationships Example

 

Why does investing money in common stock protect against loss of assets during inflation?

a. It pays higher rates of interest during inflation.

b. It provides a steady but dependable income despite economic conditions.

c. It is protected by the Federal Reserve System.                                Effective

d. It increases in value as the value of a business increases.

 

Ability to Justify Methods and Procedures Example

 

Why is adequate lighting necessary in a balanced aquarium?

a. Fish need light to see their food.

b. Fish take in oxygen in the dark. Effective

c. Plants expel carbon dioxide in the dark.

d. Plants grow too rapidly in the dark

 

4. Keep Option Lengths Similar

• Avoid making your correct answer the long or short answer

 

5. Balance the Placement of the Correct Answer

• Correct answers are usually the second and third option

 

6. Be Grammatically Correct

• Use simple, precise and unambiguous wording

• Students will be more likely to select the correct answer by finding the grammatically correct option

 

7. Avoid Clues to the Correct Answer

• Avoid answering one question in the test by giving the answer somewhere else in the test

• Have the test reviewed by someone who can find mistakes, clues, grammar and punctuation

problems before you administer the exam to students

• Avoid extremes – never, always, only

• Avoid nonsense words and unreasonable statements

8. Avoid Negative Questions

 

• 31 of 35 testing experts recommend avoiding negative questions

• Students may be able to find an incorrect answer without knowing the correct answer

 

9. Use Only One Correct Option (Or be sure the best option is clearly the best option)

 

• The item should include one and only one correct or clearly best answer

• With one correct answer, alternatives should be mutually exclusive and not overlapping

• Using MC with questions containing more than one right answer lowers discrimination between students

 

Give Clear Instructions

 

Use Only a Single, Clearly-Defined Problem and Include the Main Idea in the Question

Students must know what the problem is without having to read the response options

 

12. Avoid the “All the Above” Option

 

Students merely need to recognize two correct options to get the answer correct

 

13. Avoid the “None of the Above” Option

• You will never know if students know the correct answer

 

3.1 Writing Stems

(i) Present a single, definite statement or direct question to be completed or answered by one of the several given choices

 

A. original stem

 

Polysaccharide

are made up of thousands of smaller units called monosaccharides

are NOT found in the aloe vera leaf

are created during photosynthesis

can be described by the chemical formula: CHHOH

B. improved stem

 

Polysaccharides of the plant cell wall are synthesized mainly in the

endoplasmic reticulum

cytosol

plasma membrane

Golgi complex

 

In Example A, there is no sense from the stem what the question is asking. Example B more clearly identifies the question and offers the student a set of homogeneous choices.

(ii) Avoid unnecessary and irrelevant material in the stem. It should be clear and unambiguous

A. original stem:

 

Paul Muldoon, an Irish postmodern poet who uses experimental and playful language, uses which poetic genre in "Why Brownlee Left"?

sonnet

elegy

narrative poem

dramatic monologue

haiku

B. improved stem

 

Paul Muldoon uses which poetic genre in "Why Brownlee Left"?

 

sonnet

elegy

narrative poem

d. dramatic monologue

e. haiku

Example A contains material irrelevant to the question. This sort of material should not be used to make the answer less obvious. This tends to place too much importance on reading comprehension as a determiner of the correct option

(iii) Use clear, straightforward language in the stem of the item.

Questions that are constructed using complex or imprecise wording may become a test of reading comprehension rather than an assessment of whether the student knows the subject matter.

A. original stem

 

As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation?

 

a decrease in the workforce participation rate of women

a dispersing effect on population concentration

a downward trend in the youth dependency ratio

a broader base in the population pyramid

an increased infant mortality rate

B. improved stem

 

A major decline in fertility in a