| Home | E-Submission | Sitemap | Contact us |  
top_img
Pediatric Emergency Medicine Journal > Volume 11(3); 2024 > Article
Williams, Bretz, and Fisher: A modified Delphi study for development of an adolescent medicine curriculum for emergency medicine residents

Abstract

Purpose

More than 1,500,000 adolescents in the United States report using emergency services for the majority of their medical needs. Although the American Council for Graduate Medical Education offers some guidance on the inclusion of pediatric topics in an emergency medicine (EM) residency curriculum, it does not specifically address adolescent-related competencies. We aimed to develop a consensus on which adolescent health topics are most important to cover in a didactic curriculum for EM residents.

Methods

Physicians from multiple specialties, such as EM, pediatric EM, and adolescent medicine, were invited as panelists in a modified Delphi process. In round 1, the panelists were asked to determine which competencies in adolescent medicine are most important for a graduating EM resident to master. In round 2, they were asked to specify the competencies that should be included in an EM residency didactic curriculum. In round 3, they prioritized the most critical competencies for curriculum inclusion.

Results

Consensus was reached on a total of 26 topics that were found crucial to be included in a didactic curriculum for EM residents. Of these, the panel prioritized the following: (1) “manage a patient presenting after reported sexual assault,” (2) “accurately diagnose ovarian torsion,” and (3) “manage a patient presenting for symptoms of sexually transmitted infection.”

Conclusion

The modified Delphi process yielded recommendations to guide the development of an adolescent medicine curriculum for EM residents. Sexual and reproductive healthcare issues were identified as the most important topics.

Introduction

Entering the workforce, an emergency physician (EP) must be prepared to care for “anyone, anything, anytime,” including the unique subset of patients who straddle the line between adult and pediatric care (1). Adolescents (aged 10-24 years) often present to emergency departments (EDs) for care; over 1,500,000 United States adolescents reported that they rely on emergency services for the majority of their medical needs (2,3). Despite the recent increase in pediatric EPs, 80%-90% of pediatric emergency patients are still treated by general practitioners outside of subspecialized academic centers (4).
Unfortunately, many EPs feel the lack of skills or clinical resources to comprehensively address adolescent health issues (5). In turn, adolescents often report that they feel uncomfortable or unsafe when attempting to access emergency care (6). Because many adolescents rely on emergency services for healthcare, improving treatment in EDs is critical in combating these concerns and reducing morbidity (7). Without appropriate training, EPs incur an increased risk of adverse outcomes due to missed diagnoses, delayed diagnoses, or age-inappropriate care.
Although the American Council for Graduate Medical Education provides direction on emergency medicine (EM) residency curriculum content, the coverage of pediatric topics varies among institutions. Popular EM educational guidelines, such as the Model for Clinical Practice of EM, are not intended to address how various complaints may present differently across the lifespan (8). One large Delphi study conducted in 2017 provided further guidance on key competencies within pediatrics to supplement the Model for Clinical Practice of EM, but it did not specifically address adolescents (4). In contrast, the Society for Adolescent Health and Medicine outlines a proposed adolescent medicine curriculum. While this curriculum is comprehensive and designed for outpatient care of adolescents by subspecialists, it is not intended for use by EPs in the acute setting (9). With limited recommendations and resources, it is difficult for directors of EM residency programs to ensure that critical topics in adolescent medicine are adequately covered in the didactic curricula (10). Because most EPs practice in general EDs, rather than in academic or pediatric EDs, resident training for adolescent care is critical for all EPs.
The goal of our study was to achieve expert consensus on the adolescent health competencies that should be mastered by EPs, and to determine which of such topics should be specifically included in a didactic curriculum. In order to achieve these objectives, we employed a modified Delphi method to elicit and consolidate expert opinion.

Methods

The Delphi method is a validated and widely utilized method of determining expert agreement on competencies and generating content validity for curriculum development (11). It is a powerful tool in its capacity to include geographically diverse experts while maintaining anonymity and a relatively low cost of implementation (11,12). Redcap software (REDCap Consortium) was used to send automated email invitations and administer online surveys (13,14). All procedures and materials were approved by the institutional review board at Baylor College of Medicine (IRB no. H-51071).

1. Selection of panelists

Experts were selected based on their program leadership, prior publications, and expertise in the fields of adolescent care, EM, or graduate medical education. The resulting panel was a geographically diverse group of physicians with specialized knowledge in the fields of EM, general pediatrics, adolescent medicine, pediatric EM, pediatric and adolescent gynecology, and family medicine. General practitioners were intentionally over-represented because most adolescents are treated by generalists in EDs (4). To capture the expertise of physicians who care for adolescents outside academic medical centers, the survey link was also disseminated to practicing EPs in the community, resulting in the participation of an additional 8 non-academicians. Panelists completed a 3-round modified Delphi study.

2. Study design

In round 1, we asked the panelists how important it is that a graduating EM resident attain specific competencies using a candidate list of 27 topics relevant to adolescent care in EDs (Table 1). These topics were identified by a thorough review of the adolescent health literature, as well as the existing curriculum composed by the Society for Adolescent Health and Medicine (9). To assess the importance, the panelists used a 5-point scale (1, not important; 2, minimally important; 3, somewhat important; 4, very important; and 5, extremely important). Additionally, they were asked to provide free-text responses to suggest additional topics. Only topics proposed by at least 2 different panelists were advanced to round 2.
In round 2, the panelists were asked to rate the importance for an EM residency curriculum to include a didactic instruction on each topic. In round 3, they were provided the list of topics that met the consensus in round 2 and asked to prioritize 10 topics that should be included in a didactic adolescent medicine curriculum for EM residents (Table 2).

3. Data analysis

In all 3 rounds, the composition of the panel was assessed to ensure its representativeness and diversity in terms of specialty, community and academic practice setting, and regional location. No individual responses were linked with identifiable information, in keeping with widely accepted standards for publications using a Delphi model (15,16). To maintain the panelists’ anonymity, each round of the survey was sent to all, a strategy that has been shown to produce results consistent with a “respondents-only” approach, wherein subsequent surveys are distributed only to the experts who responded in previous rounds (17). For those recruited from the community, we did collect email addresses to distribute the remaining surveys; this information was not linked to individual responses. Descriptive statistics were calculated for each round using the Redcap software, including the percentage of the panelists who assigned each topic either a 4 or 5 on the 5-point scale (13,14). We defined a priori consensus criterion as ≥ 75%.

Results

An overview of the Delphi process is depicted in Fig. 1. Sixteen invited experts participated in round 1, in addition to 8 community physicians, yielding a total panel size of 24 experts. Round 1 panelists had a diverse range of relevant specialties. Appendix 1 (https://doi.org/10.22470/pemj.2024.00955) details the composition of the panel in each round, which included nationwide physicians.
In round 2, the panelists evaluated a total of 26 topics, including the 23 that met consensus criterion in round 1 as well as additional 3 that were proposed by at least 2 panelists in round 1 (Table 1). Twenty-four of the 26 topics reached consensus as either a 4 or 5 on the 5-point scale to be included in a didactic curriculum. Two topics (“treat overuse injuries in adolescents” and “offer counseling on substance use cessation”) did not meet consensus criterion in round 2 (Table 1).
Finally, in round 3, a total of 17 panelists prioritized the most important topics. The following topics were most often selected as being integral to a didactic adolescent medicine curriculum for EM residents: 1) “manage a patient presenting after reported sexual assault,” 2) “accurately diagnose ovarian torsion,” and 3) “manage a patient presenting for symptoms of sexually transmitted infection (Table 2).”

Discussion

No standard set of competencies or curriculum exists for EM residents on the care of adolescent patients (10). To our knowledge, this study is the first to specify which emergency adolescent care-related competencies should be included in a didactic curriculum for EM residents. Twenty-six unique topics were identified by the panelists. Given structural and time constraints, as well as the tremendous volume of material that must be covered during EM residency training, the panelists prioritized the most important topics. Based on our recommendations, directors of residency programs may modify their specific didactic offerings depending on their individual needs and resources.
The top 3 topics are related to sexual, reproductive, or genitourinary issues. Approaching these topics with adolescents is common but may be more uncomfortable than with adults, requiring a different approach. Open conversation and collaboration between patients and physicians are particularly consequential in the current landscape of rapidly changing legislation regarding limited reproductive care in many states in the U.S. In a future curriculum, it will be important to emphasize how to communicate when discussing sexual health topics with adolescents, such as navigating issues of consent and confidentiality for minor patients. Although the competency of “effectively tailor communication strategies to developmental level” was not frequently (12.5%) ranked in round 3, the panelists might feel that it was covered with greater specificity by other proposed topics, such as “manage a patient presenting for symptoms of sexually transmitted infection.”
“Manage a patient presenting for a mental health complaint” was chosen as a top priority by the panelists. This finding might be related to the reality that adolescent mental health in the U.S. has become an increasing concern in the last several years. Since the beginning of the coronavirus disease 2019 pandemic, the Centers for Disease Control and Prevention have reported the development of “an accelerating mental health crisis among adolescents (18).” Approximately 1 in 4 adolescents will experience a psychiatric disorder that results in severe impairment, and about half of all lifetime cases of psychiatric illness begin by 14 years of age (19,20). Because mental health disorders often present differently in adolescents, it is crucial to address these concerns specifically in the population. Adolescents are also at increased risk for the onset of many mental health conditions, requiring providers to appropriately recognize a previously undiagnosed condition. Meanwhile, fewer physicians selected related topics such as “treat medical complications of eating disorders” or “manage a patient presenting with non-suicidal self-injury.” One possible explanation for this finding may be that panelists believed these specific skills were not significantly different than those needed to treat adults. Alternatively, panelists may have considered these topics to be included in the broader ability to “manage a patient presenting for a mental health complaint.”
Only 2 of 26 topics met the consensus criterion in round 1, but not in round 2 of the Delphi process. This discrepancy might be attributed to a slight difference in panel composition between the rounds. It may also have been that some panelists felt EPs should be competent in these areas (round 1) but not that they were important to cover specifically in a didactic curriculum (round 2). Nevertheless, this level of concordance is satisfactory and serves to indicate the reliability of our findings (21).
This study has its limitations. A low response rate is common in conducting a Delphi process, which we attempted to mitigate by sending 2 rounds of follow-up email reminders to invite experts (22). EPs showed a high response rate, perhaps because they felt the material was most relevant to their own practice and teaching responsibilities. As in all Delphi studies, it is possible that a different composition of panelists would lead to a different result, though this panel composition is representative of EPs treating adolescents in EDs. Finally, we recognize that a didactic adolescent medicine curriculum may be insufficient to fully address this particular educational gap, which likely contributes to the discomfort of both physicians and adolescents in EDs. However, it is beyond the scope of this study to suggest additional simulation or clinical training opportunities.
In this modified Delphi process, our expert panel agreed on a set of 26 competencies in adolescent medicine to include in a didactic curriculum for EM residents and prioritized the top 10 of these findings. In the future, this guidance can supplement that provided by the EM Model of Clinical Practice to offer a content blueprint for the development of a curriculum specific to emergency adolescent care. Considering the unique needs of the adolescent population and the current lack of specific guidance, our findings provide a crucial step in the effort to adequately prepare EPs for the unique challenges of treating adolescents.

Notes

Author contributions

Conceptualization, Resources, and Methodology: all authors

Data curation, Formal analysis, Investigation, Project administration, and Visualization: EW and KF

Supervision: KF

Writing-original draft: EW

Writing-review and editing: all authors

All authors read and approved the final manuscript.

Conflicts of interest

No potential conflicts of interest relevant to this article were reported.

Funding sources

No funding source relevant to this article was reported.

Fig. 1.
An overview of the Delphi process.
pemj-2024-00955f1.jpg
Table 1.
Results from Delphi rounds 1 and 2: expert agreement that an emergency medicine resident should attain a given competency or that it be included in a didactic curriculum
Curricular content Round 1 Agreement: Perform, % Round 2 Agreement: Curriculum, %
Recognize problematic substance use 100 86.6
Manage a patient presenting for a mental health complaint (e.g., depression, suicidal ideation, anxiety, or panic attacks) 100 93.3
Offer counseling on resources available to an adolescent in an unsafe domestic situation 100 80.0
Identify patients at high risk for future gun- or violence-related injury 100 93.3
Treat acute intoxication due to a legal substance 94.5 93.4
Treat acute intoxication due to an illegal substance 94.5 93.4
Manage a patient presenting for symptoms of sexually transmitted infection 94.5 100
Recognize signs of intimate partner violence 94.5 100
Treat conditions that are disproportionately common in adolescent patients (e.g., testicular torsion or pneumothorax) 94.5 100
Understand local regulations regarding age of consent for treatment for different conditions 94.4 100
Manage a patient presenting with non-suicidal self-injury 94.4 86.7
Treat medical complications of eating disorders 94.4 80.0
Identify potential victims of human trafficking 94.4 93.3
Treat an acute injury sustained during sports activities 94.2 100
Manage a patient presenting after reported sexual assault 94.1 100
Manage a patient presenting with a request for emergency contraception 88.9 93.3
Provide nonjudgmental patient-centered care for patients who identify as members of the lesbian/bisexual/gay/transgender/queer community 88.9 100
Navigate electronic health records and billing to protect confidentiality, as allowed by state law 88.9 86.7
Treat the medical sequelae of a suicide attempt 88.9 93.4
Manage a patient presenting with high suspicion for human trafficking 88.9 100
Treat acute exacerbations of congenital conditions (e.g., cystic fibrosis or congenital heart disease) 83.4 93.3
Treat overuse injuries in adolescents (e.g., patellofemoral pain syndrome, shoulder instability, tibial stress fracture, or spondylolysis) 83.3 53.3
Offer counseling on substance use cessation 77.8 60.0
Offer counseling regarding routine use of contraception 72.2 NA*
Provide information regarding local youth violence-prevention programs 72.2 NA*
Offer counseling regarding an unplanned pregnancy 66.6 NA*
Provide information on local regulations and resources for gender-affirming care 50.0 NA*
Manage a patient presenting for initial evaluation of heavy menses resulting in symptomatic anemia NA 93.4
Accurately diagnose ovarian torsion NA 100
Effectively tailor communication strategies to developmental level NA 100

* Topic that failed to meet consensus in round 1.

Topic proposed by experts in round 1.

Table 2.
Results from Delphi round 3: key topics of an adolescent medicine curriculum for emergency medicine residents
Topic Agreement: This is one of the top 10 topics, %
Manage a patient presenting after reported sexual assault 87.5
Accurately diagnose ovarian torsion 75.0
Manage a patient presenting with symptoms of sexually transmitted infection 75.0
Manage a patient presenting for a mental health complaint (e.g., depression, suicidal ideation, anxiety, or panic attacks) 68.8
Treat conditions that are disproportionately common in adolescent patients due to physiologic development (e.g., testicular torsion or pneumothorax) 68.8
Provide nonjudgmental patient-centered care for patients who identify as members of the lesbian/bisexual/gay/transgender/queer community 56.3
Treat acute intoxication due to a legal substance 56.3
Treat acute intoxication due to an illegal substance 56.3
Treat an acute injury sustained during sports activities 56.3
Understand local regulations regarding the age of consent for treatment for different conditions 50.0
Manage a patient presenting for initial evaluation of heavy menses resulting in symptomatic anemia  37.5
Manage a patient presenting with a request for emergency contraception 37.5
Treat acute exacerbations of congenital conditions (e.g., cystic fibrosis or congenital heart disease) 37.5
Recognize signs of intimate partner violence 31.3
Treat the medical sequelae of a suicide attempt 31.3
Manage a patient presenting with high suspicion of human trafficking  25.0
Navigate electronic health records and billing to protect the confidentiality, as allowed by state law  25.0
Recognize problematic substance use 25.0
Treat medical complications of eating disorders 25.0
Identify potential victims of human trafficking  18.8
Effectively tailor communication strategies to the developmental level 12.5
Manage a patient presenting with non-suicidal self-injury 12.5
Offer counseling on resources available to an adolescent in an unsafe domestic situation 12.5
Identify patients at high risk for future gun- or violence-related injury  6.3

References

1. Zink BJ. Preface. In: Zink BJ. Anyone, anything, anytime: a history of emergency medicine. Mosby Elsevier; 2006. p. 6-11.

2. Sawyer SM, Azzopardi PS, Wickremarathne D, Patton GC. The age of adolescence. Lancet Child Adolesc Health 2018;2:223–8.
crossref pmid
3. Weiss AL, D’Angelo LJ, Rucker AC. Adolescent use of the emergency department instead of the primary care provider: who, why, and how urgent? J Adolesc Health 2014;54:416–20.
crossref pmid
4. Mitzman J, King AM, Fastle RK, Hopson LR, Hoyle JD Jr, Levasseur KA, et al. A modified Delphi study for development of a pediatric curriculum for emergency medicine residents. AEM Educ Train 2017;1:140–50.
crossref pmid pmc pdf
5. Goyal M, Zhao H, Mollen C. Exploring emergency contraception knowledge, prescription practices, and barriers to prescription for adolescents in the emergency department. Pediatrics 2009;123:765–70.
crossref pmid pdf
6. Northern Ireland Commissioner for Children and Young People (NICCY). Still waiting: a rights based review of mental health services and support for children and young people in Northern Ireland [Internet]. NICCY; 2018 [cited 2021 Nov 13]. Available from: https://www.niccy.org/wp-content/uploads/media/3114/niccy-still-waiting-report-sept-18-web.pdf.

7. Alele FO, Emeto TI, Callander EJ, Watt K. Non‐urgent paediatric emergency department presentation: a systematic review. J Paediatr Child Health 2019;55:271–7.
crossref pmid pdf
8. Beeson MS, Bhat R, Broder JS, Gaeta TJ, Janssen A, Karl ER, et al. The 2022 model of the clinical practice of emergency medicine. J Emerg Med 2023;64:659–95.
crossref pmid
9. Society for Adolescent Health and Medicine (SAHM). Adolescent medicine resident curriculum [Internet]. SAHM; 2020 [cited 2023 Mar 23]. Available at: https://adolescenthealth.org/training-and-cme/adolescent-medicine-resident-curriculum/.

10. Williams ES, Guerrero N, Sisson A, Fisher KM. Assessing the gap in adolescent emergency care training for emergency medicine residents: a systematic review. Cureus 2023;15:e40814.
crossref pmid pmc
11. Ludlow J. Delphi inquiries and knowledge utilization. In: Linstone HA, Turoff M, editors. The Delphi method: techniques and applications. Addison-Wesley Publishing Co.; 1975. p. 97-118.

12. Humphrey-Murto S, Varpio L, Wood TJ, Gonsalves C, Ufholz LA, Mascioli K, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med 2017;92:1491–8.
crossref pmid
13. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.
crossref pmid pmc
14. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software partners. J Biomed Inform 2019;9:95103208.

15. Witkin BR, Altschuld JW. Specialized survey and group techniques for data gathering and analysis. In: Witkin BR, Altschuld JW. Planning and conducting needs assessments: a practical guide. Sage Publications, Inc.; 1995. p. 186-208.

16. Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Inf Manag 2004;42:15–29.
crossref
17. Boel A, Navarro-Compán V, Landewé R, van der Heijde D. Two different invitation approaches for consecutive rounds of a Delphi survey led to comparable final outcome. J Clin Epidemiol 2021;129:31–9.
crossref pmid
18. Balingit M. ‘A cry for help’: CDC warns of a steep decline in teen mental health: more than 4 in 10 told the health agency they felt ‘persistently sad or hopeless’ [Internet]. The Washington Post; 2022 Mar 31 [cited 2024 May 8]. Available from: https://www.washingtonpost.com/education/2022/03/31/student-mental-health-decline-cdc/.

19. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2010;49:980–9.
crossref pmid pmc
20. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005;62:617–27.
crossref pmid pmc
21. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008–15.
crossref pmid
22. Hsu CC, Sandford BA. Minimizing non-response in the Delphi process: how to respond to non-response. Pract Assess Res Eval 2007;12:17.

Appendices

Appendix 1

Panel compositions

Emergency medicine: academic Emergency medicine: community General pediatrics Adolescent medicine Pediatric emergency medicine Pediatric and adolescent gynecology Family medicine Geographic distribution
Round 1* 7 8 0 1 1 2 1 NE, 1; MW, 10, south, 0; west, 0; and NS, 13
Round 2 6 3 1 1 0 2 1 NE, 4; MW, 7; south, 2; west, 1; and NS, 0
Round 3 6 4 2 2 2 1 0 NE, 3; MW, 9; south, 3; west, 2; and NS, 0

* Four panelists in round 1 did not indicate their specialties.

NE: northeast, MW: Midwest, NS: not specified.

Editorial Office
The Korean Society of Pediatric Emergency Medicine
Chungmu-building office 213, 197, Toegye-ro, Jung-gu, Seoul 04557, Korea
TEL: +82-2-3674-7888   FAX: +82-2-3674-7889   E-mail: editor@pemj.org
About |  Browse Articles |  Current Issue |  For Authors
Copyright © Korean Society of Pediatric Emergency Medicine.                 Developed in M2PI
Close layer
prev next