Unlike quantitative disciplines, like math or calculus where the expected outcome of manipulating two variables is 100% predetermined, healthcare isn’t an exact science. Dynamic variables, such as patient physiology, psychology, genetic predispositions, unpredictable drug reactions, team (doctors, surgeons, pathologists, nurses) dynamics, and care-providing environment, come together to determine patient outcomes.
The Case for Scenario-Based Assessments
So, when healthcare professionals learn a new procedure or train on cutting-edge equipment, it’s important to assess the quality of their learning over quantity. Delivered on their own, true/false quizzes and multiple-choice assessments typically test for specific responses.
For instance, an assessment question such as this:
would certainly test learner knowledge levels. However, devoid of broader context – which is the hallmark of scenario-based testing – this type of assessment does not consider the multitude of “dynamic variables” discussed earlier. Because healthcare professionals do not, typically, deal with situations that have binary outcomes, instructional designers need a more appropriate assessment methodology. That’s why scenario-based assessments that integrate multiple choice with situational use case scenarios are so critical for healthcare eLearners.
Creating Scenario-Based Assessments
Assessment feedback, from guessing a wrong answer to the above question, will help learners review/revise their training content for the next test. But there is limited scope for learners to learn from the consequences of their mistakes. To do that, instructional designers must provide significant context (stories) around the assessment, so learners can picture themselves in specific healthcare delivery situations as part of the evaluation.
Here are some tips that instructional designers can leverage to create effective scenario-based assessments.
1. Consult with a multi-disciplinary focus group when developing scenarios.
Before putting your assessments together, make sure you engage in broad-based consultations with cross-disciplinary teams. The teams could include front-line healthcare professionals and hospital administrators, HR and training staff, as well as (potentially) support staff.
Although this is part of the well-known “know your audience” phase of developing instructional content, it can sometimes be a challenge identifying the right stakeholders. Sometimes, who you leave out from the consultative process might have a greater impact on determining the effectiveness of the assessment scenarios, than who you consult with.
Here’s an example of a scenario that assesses learner appreciation for broad-based consultation. In this case, had the design team taken a “total environment of care” approach, and listened to the voice of laundry room staff, janitorial and other support services, their needs and concerns would have factored into room design and staffing patterns.
2. Map out the clinical scenarios for assessment
Before commencing detailed assessment development, instructional designers must map out all the scenarios that require assessment. This mapping could potentially come as a result of the broad-based consultative process highlighted above. Mapping must include:
- High-level description of the scenario
- Details of the problem or skills assessed, critical challenges, decisions or actions tested/assessed, possible outcomes of the scenario
- Identify the key protagonists and supporting characters for the scenario
- Set out the environment (roadside accident, hospital ER, physician’s office)
- Relevant Key-feature Questions (KFQs) that the assessment scenarios are built around
As part of the assessment-mapping process, you must ensure that your assessments span the entire curriculum. Creating a scenario matrix, such as the one illustrated in the figure below, is a good way to map scenarios to skills in preparation for building the assessments.
This mapping exercise will provide instructional designers the roadmap for creating effective scenarios. Training developers can then ensure that appropriate assessments are in place to support the learning outcomes of each of them.
3. Focus the assessment scenario on real-life challenges
We’ve all seen scenario-based assessments that are based on hypothetical situations. These really don’t add value to evaluating learning outcomes. When developing your assessment scenarios, focus on challenges and situations that healthcare workers are likely to face in their day-to-day care-providing environment.
For example, developing a scenario that assesses an urban ER physicians’ ability to triage a mass-casualty (car crash) situation may be more appropriate than assessing their ability to triage and treat multiple shark attack victims.
4. Make the assessment scenarios as specific as possible
When assessing cultural sensitivity skills in healthcare delivery, for instance, you could pose generic questions that apply to a cross-section of culturally appropriate situations. However, there is more value in designing scenarios that seek to assess specific and precise knowledge areas.
Doctors, surgeons and nursing staff who are less familiar with Asian traditions (or those of other cultures) will learn more from the above scenario than they would takeaway had this scenario been more broad-based.
5. Use branching scenarios for better assessments
The best way to describe branching scenarios is scenarios within scenarios.
Instructional designers must develop multiple decision points within each scenario, and give learners options to go down different paths that lead to sub-scenarios.
Each choice that the learner makes, must bring him/her closer to addressing the core challenge of the scenario. In our multiple car crash scenario alluded to earlier, for instance:
- The main scenario can depict a multi-vehicle accident scene
- One sub-scenario may branch to assessing abilities to triage the scene
- Another sub-scenario can assess the learner’s ability to deal with excessive bleeding
- Additional assessments may cover scenarios that depict head trauma, shock or patients in coma
The objective of branching scenarios is to gradually eliminate the number of possible outcomes with each selected branch. This assesses the learner’s ability to analyze the macro picture (a multi-vehicle crash scene), and then home in on specific treatment protocols (head trauma). Instructional designers may find using storyboarding tools, such as LucidChart, Canva or Adobe Artboards, a useful resource to plan and design branching scenarios.
6. Create multi-skill/multi-role scenarios
Healthcare workers typically work in a team environment. Therefore, what one provider does (or omits to do!) makes a difference in the level of care delivered by another. To ensure that the assessments mimic this situation, instructional designers should design scenarios that assess the capabilities of multiple roles and multiple skills that interact with each.
For the above scenario, you may create assessments to determine whether the team –nursing staff and physician – observed proper communication protocols; and how to better improve communication amongst various team members (e.g.: The nurse always repeats or summarizes what the physician directs him/her to do)
Leave A Comment