Introducing AI to the Emergency Management Workforce: A Case Study

Artificial Intelligence (AI) is not coming to the emergency management (EM) workforce – it is already here and has been for a while. For example, every time an online search is done using a standard browser, its embedded AI tool searches through the web to provide you with the answer you most likely want. If you have allowed it to do so, the search engine will also learn from engagement over time to refine future searches’ answers and push targeted information (e.g., articles or products for sale) based on its continually evolving profiles. This article provides a case study on how a generative AI-based lesson was incorporated to address an EM training course experiencing a widening gap in student outcomes generated by some students’ unstructured use of AI.

In this program, faculty observed some student groups completing their work much more quickly and with a wider variety of content. In contrast, others struggled to complete the assignment in the allotted time. The difference was the use of the modern version of high school’s “CliffsNotes” but instead of the printed yellow and black printed study guides, these students were using ChatGPT’s GPT-4o generative AI tool to create a dynamic guide that answered their specific questions on demand. Recognizing the “culprit,” faculty had a choice: ban or embrace its use. Considering the pro-adoption trends in EM and education, the choice was clear: A student-led activity was created to build AI competencies that successfully elicited full engagement and improved content quality.

Getting to the Bottom of Student Outcomes

This case study was conducted during a 2024 National Emergency Management Advanced Academy training program sponsored by the Emergency Management Professional Program through the Federal Emergency Management Agency’s National Disaster and Emergency Management University. The program delivers four one-week sessions in a forced-sequential cohort model. It was a local delivery with approximately 20 participants from several U.S. states. The target population was current mid- to late-career emergency management professionals with five or more years of experience and who led teams. The curriculum outcomes were designed to develop individual and team leadership skills and improve the EM profession by teaching students to use current research to create new and innovative best practices, leading the profession forward using the program’s “Research, Synthesize, Apply” model.

The cohort’s students were leaders in various private and public EM specialties, including university, transportation, hospital, military, first response, voluntary agencies, and traditional county and state government EM. Three faculty members were assigned and remained with the cohort for all four sessions. Faculty assigned students to groups, and faculty members changed the membership of those groups for each course. Observation during the first session showed some groups were finishing work much earlier than others. As faculty observed the first three sessions, they identified specific individuals as common denominators in groups that consistently finished more quickly and had more focused products during activity reports. Upon analysis, the difference was their use of AI.

Faculty observations were that, while the final products were more focused and the content was created more quickly, not everyone in the group had fully participated. Using AI allowed some students to craft their products from a more extensive research dataset and to hone in on the most relevant facts more quickly, moving them from the research to the synthesis stage before their peers. The result was that the “analog students” (i.e., those using traditional online search engines and reading the entirety of the material to glean the pertinent points) disengaged to avoid holding back the process. It was observed that they were unprepared to engage in the synthesis portion without the time to read the materials. They only began re-engaging with the group in the final product’s editing and presentation stages. Faculty recognized that AI was both a problem for some learners and a solution for others.

Innovating Learning With AI

The faculty introduced a learning activity in Week 4 to address this. The standard module content was designed around “leading up” (i.e., influencing those at higher organizational levels). The activity was AI-focused, using ChatGPT’s GPT-4o to reinforce the module topic and thread other leadership themes from the previous three weeks.

The lesson plan was that first, one student who had been particularly active in using AI to complete assignments was asked to lead a whole-class session on what AI could do and how to use it for researching EM problems. He had been observed consistently throughout the three weeks, providing AI-generated information to his groups, which proved its value. However, others had not learned to do the work themselves; they “drafted” off his expertise. In this activity, he introduced his classmates to the processes behind his work. Faculty supported his authority on the topic by wholly distributing the activity’s leadership (i.e., content, design, and presentation) to him and only minimally engaged to ensure learning objectives were met within the allotted time.

Other students were animated in their questioning and excited about the opportunity to try AI. Research by Miller and Rockabrand has shown that exposing people to AI technology can make them more inclined to use it, which is also evident here. The student leader demonstrated “prompt engineering,” which is designing and refining precise and effective instructions or questions to guide the tool in generating accurate and relevant responses. Prompt engineering is a foundational skill for people learning to use AI tools, and poor prompting is often a frustrating barrier for new users because poor prompts lead to poor results. After the demonstration, students practiced designing and refining prompts in a participatory large-group activity using ChatGPT’s GPT-4o  to comprehensively summarize the introduction to Niccolo Machiavelli’s The Prince.

Aside from practicing iterative prompt engineering, the goal of this activity was to understand this work’s full historical and socio-political context in natural language. The Prince was chosen because it is a short book and a foundational text in leadership studies. One of Machiavelli’s purposes in writing was to “lead up,” which aligned with the module content. It is translated from Renaissance Italian, making the text more difficult, as the language, even in translation, is formal and somewhat convoluted for the modern English reader.

Once the introduction was completed, the groups were shown how to limit the AI client’s results to a specific body of knowledge (i.e., instead of the whole internet) by putting the entire book into the ChatGPT interface. This is one way to limit “hallucinations” or false, biased, or misleading information presented as facts. This occurs because the software cannot distinguish the quality of the information; it can only repackage and present the information it finds in the training data. If the training data is restricted to factual or relevant data, the output reflects that.

The students were then tasked with using ChatGPT to find discussions about “leading up” within the text (i.e., research). Groups were directed to read the original text for context and see the point Machiavelli made in those sections contextualized for his time (i.e., synthesize) using the information from the first activity. Finally, they were asked to apply those points to modern EM problems, negative or positive, and to share what they learned.

After this exercise, analog students understood how to use the technology that enabled them to search an entire book quickly to find relevant portions. Having the material presented in natural language allowed them to grasp “plain English” meaning and context before re-reading the original language to get subtle nuances. An additional bonus of this exercise was that the avid AI users had typically not returned to the original text, and this lesson provided a model to help them expand their comprehension by finding nuances in an author’s original writing.

Additionally, the whole group became more effective as they more evenly distributed the workload for fuller participation. The analog students in this case study generally had more professional experience than the AI users. Because of the speed of AI, their practical experience was often not incorporated because the AI users had moved on to the synthesis stage before the analog readers had comprehended the base content. By teaching a structured use of AI, speeding up the analog readers, and slowing down the AI readers, groups worked more cohesively, blending technology and experience to complete the assignment in a novel way that helped each group learn more collaboratively. This became evident when the final product (i.e., apply) reflected more of the professional experiences of the analog learners.

In post-activity reviews, students appreciated the opportunity to learn about AI. Many had heard negatives about it and were unsure how it worked and whether it was worth the effort. The reviews of the activity were generally positive in post-activity discussions and anonymous written evaluations. Similar results were achieved in additional small-scale tests of this curriculum in other learning environments.

AI Competency in EM

This type of AI use in training is foundational to building competency and comfort around AI technology for the existing EM workforce. The barriers to entry are minimal and consist mainly of instructional design time. Professional EM educators and trainers see a near future in which AI will build from these basic activities into more advanced and innovative uses, enabling individualized content and increasingly sophisticated user experiences. Users will engage with AI using natural language communication, simulations, gamification, etc., to assess areas where each learner is strong or struggling. This allows training and education to be tailored to the needs of each learner (i.e., competency-based education), which is reemerging as a leading model for future educational initiatives, possible now because of the technology’s ability to assess students and deliver individually-tailored content at a speed that human teachers cannot duplicate.

AI can also be paired with other educational initiatives to increase their effectiveness, as seen when paired with open education resources that use open content licenses to create individualized content, as was done with this assignment. (The age of The Prince means that copyright laws do not restrict it, and it is therefore considered open source.) Regardless of future applications, the result is that AI will create competency and outcomes-based learning tailored to each learner’s unique needs and learning style preferences while being more efficient for the learner and their employer.

BONUS: If you would like to use this activity, download the lesson plan.