Converting to Scenario-Based Questions

Many of the traditional multiple choice questions we use in assessment are abstract and measure only whether people recall facts they heard in the last 5 minutes. Converting these questions to scenario-based questions can increase the level of difficulty, measure higher level skills, and provide relevant context. One-question mini-scenarios can improve your assessments and practice activities.

Example question

Let’s say you’re creating training for managers on how to provide reasonable accommodations for employees. You drafted a set of traditional multiple choice questions as a quiz for the end of the course, but they’re all very low level. You want to improve the quality of your assessment with some scenarios.

This is a question from your current quiz that measures recall of a fact from the training. The rest of the assessment is similar.

Example 1: Original question

What reasonable accommodation is recommended for a temporary disability or medical issue affecting work?

None; reasonable accommodations are only used for permanent or long-term disabilities.
Unpaid time off can be offered as an accommodation for temporary issues.
Paid time off should be offered, even if it exceeds the amount of paid time off other employees receive.
Schedule adjustments are the only accommodation offered for temporary issues.

Align to objectives

What are your objectives? Does your assessment align to them? If not, rewrite it.

In this example, the objective is “Managers will follow the company’s policies and procedures for providing reasonable accommodations for disabilities.” The objective is application level; you need to apply this procedure.

The question assesses recall; the objective requires application. Therefore, this question should be rewritten at a higher level.

When would people use this?

The first step to shifting from traditional to scenario-based questions is asking when people would use the information.

When would managers need to know about handling temporary disabilities? A common situation would be due to an illness or surgery. Maybe an employee needs a reduced schedule due to fatigue from chemo. Maybe an employee needs time off to recover from back surgery.

For each multiple choice question, ask yourself how learners would use that information on the job. When would they need to differentiate between those options?

If you can’t come up with any situation in which people would need this information on the job, why are you asking that question? If you have a question with just irrelevant information, skip down to the section on complete rewrites below.

Set the scene: scenario as introduction

One method to revise the question is to add a scenario to introduce the choices. This provides context. It shifts the question from just recalling information to using that information to make a decision.

Let’s see how this works with the previous example. The scenario introduces the question. The choices are essentially the same as before, but now it’s a decision about how to work with an employee you manage. Instead of measuring recall, this question measures if learners can apply the reasonable accommodations policy.

Example 1: Revised scenario-based question

Simon, a graphic designer on the team you manage, is having surgery. He requested 2 weeks time off to recover after his surgery. What is the best response, according to our policies?

Let Simon know he can use his accrued vacation time. Reasonable accommodations are only used for permanent or long-term disabilities.
Provide two weeks unpaid time off as an accommodation.
Allow Simon to adjust his schedule as long as he still works his total normal hours.

Notice that this scenario-based question isn’t long; it’s only 2 more sentences than the original question. The brief scenario provides context for decision-making. The alternatives are similar to the originals, but with some specific details to fit the scenario context.

Another example scenario question

Here’s another example of a traditional multiple-choice question rewritten as a scenario-based question.

Objective: Managers will follow Fair Labor Standards Act (FLSA) pay and overtime regulations to ensure accurate employee compensation.

Example 2: Original question

Under the Fair Labor Standards Act (FLSA), what is the threshold for when overtime pay typically begins for non-exempt employees?

8 hours in a single workday
40 hours in a work week
80 hours in a two-week pay period

As with Example 1, this isn’t a terrible multiple choice question. It’s just abstract and focused on comprehension rather than application.

Example 2: Revised scenario-based question

Manuel is an assistant manager for Retail Spree, a discount retailer.

Lisa, one of the employees, asks, “I’m picking up an extra 6-hour shift on Saturday. I already worked 36 hours this week. Do I track the whole shift as overtime?” How should Manuel respond to follow FLSA regulations?

Just track it as regular hours, since you get paid biweekly, and it will even out next week.
Yes, report the entire shift as overtime since that’s over 40 hours for the week.
No, report 4 hours as regular time and 2 hours as overtime.

This version provides some context in the form of an employee question, which makes it more realistic and relevant to the audience’s job. If you had a different audience, you’d want to change the details to make it relevant to their setting.

Complete replacement

Sometimes adding a scenario at the beginning won’t work, and you need a complete rewrite of the question. If the question is something unrelated to your objectives or that people will never use on the job, you have to start over and replace the question.

Look at this example. Would a manager ever need to know this history on the job? Will they be more effective at offering accommodations if they can memorize this date?

Example 3: Original

In what year was the Americans with Disabilities Act or ADA passed by Congress?

1985
1990
1995
2000

We’ve all seen questions like this on quizzes before. They’re easy to write, but they don’t assess anything meaningful. Replacing it with a scenario-based question would give you a more valid assessment.

Example 3: Replacement

One of your employees, Miranda, brought documentation from her ophthalmologist about her vision and how it affects her driving. Her night vision is deteriorating. Miranda has requested a change in her work schedule. She wants to start and end her work day later to avoid driving in the early morning when it’s still dark. What is the best response?

Agree to adjust Miranda’s schedule.
Tell Miranda to contact HR to start the official accommodation process.
Tell Miranda that the schedule change is not possible since it creates too much burden on the rest of the team.
Ask Miranda if any other accommodations would work instead of a schedule change.

What do you want to learn?

What else would you like to learn about writing these kinds of scenario-based questions? Do you have questions I could answer in a future post? Let me know in the comments or by replying to this email.

For more information, check out these related posts:

Mini-Scenarios for Assessment
My collection of 70+ posts on Storytelling and Scenarios

Originally published on 8/1/2017 with the title “Converting Traditional Multiple Choice Questions to Scenario-Based Questions.” Updated and republished with a new title 8/4/2020. Updated with additional examples 11/27/2024.

Are you on Bluesky?

Like many L&D folks, I’ve started to post more on Bluesky. This feels like a good time to join; I see some critical mass of people in our field for conversations. You can follow me at christytucker.bsky.social. Mark Sheppard’s Learning & Development Starter Pack is a great starting point to find other voices to follow.