TY - JOUR
T1 - Identifying Gaps in the Performance of Pediatric Trainees Who Receive Marginal/Unsatisfactory Ratings
AU - Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN) Validity of Resident Self-Assessment Group
AU - Li, Su Ting T.
AU - Tancredi, Daniel J.
AU - Schwartz, Alan
AU - Guillot, Ann
AU - Burke, Ann
AU - Trimm, R. Franklin
AU - Guralnick, Susan
AU - Mahan, John D.
AU - Gifford, Kimberly A.
AU - Basila, Dennis
AU - Dougherty, Katherine
AU - Robus, Richard S.
AU - Matrone, Christa
AU - Schumacher, Daniel J.
AU - Dewar, Stephanie B.
AU - Hofkosh, Dena
AU - Lieberman, Rhett
AU - Poynter, Sue E.
AU - Tuell, Dawn
AU - Huff, Janara J.
AU - Rincon, Marielisa
AU - Sease, Kerry K.
AU - Yeh, Sylvia H.
AU - McKenna, Michael P.
AU - Price, Laura
AU - Donnelly, Kathleen M.
AU - Carter, Meredith L.
AU - Rappaport, David I.
AU - Wittler, Robert R.
AU - Frey-Vogel, Ariel
AU - Scott-Vernaglia, Shannon E.
AU - Perkins, Kate
AU - Carson, Savanna
AU - Kapklein, Matthew J.
AU - Unti, Sharon M.
AU - Ferrell, Cynthia L.
AU - Dunbar, Alston E.
AU - Bhavaraju, Vasudha L.
AU - Caputo, Grace L.
AU - Kramer, Jane E.
AU - Miner, Michelle
AU - Calaman, Sharon
AU - Cruz, Mario
AU - Spector, Nancy D.
AU - Barone, Stephen R.
AU - Camp, Tammy
AU - Elliott, Sean P.
AU - Franke, Hillary A.
AU - Fagan, Heather
N1 - Publisher Copyright:
Copyright © by the Association of American Medical Colleges.
PY - 2018/1/1
Y1 - 2018/1/1
N2 - Purpose To perform a derivation study to determine in which subcompetencies marginal/unsatisfactory pediatric residents had the greatest deficits compared with their satisfactorily performing peers and which subcompetencies best discriminated between marginal/unsatisfactory and satisfactorily performing residents. Method Multi-institutional cohort study of all 21 milestones (rated on four or five levels) reported to the Accreditation Council for Graduate Medical Education, and global marginal/unsatisfactory versus satisfactory performance reported to the American Board of Pediatrics. Data were gathered in 2013-2014. For each level of training (postgraduate year [PGY] 1, 2, and 3), mean differences between milestone levels of residents with marginal/unsatisfactory and satisfactory performance adjusted for clustering by program and C-statistics (area under receiver operating characteristic curve) were calculated. A Bonferroni-corrected significance threshold of.0007963 was used to account for multiple comparisons. Results Milestone and overall performance evaluations for 1,704 pediatric residents in 41 programs were obtained. For PGY1s, two subcompetencies had almost a one-point difference in milestone levels between marginal/unsatisfactory and satisfactory trainees and outstanding discrimination (≥ 0.90): organize/prioritize (0.93; C-statistic: 0.91) and transfer of care (0.97; C-statistic: 0.90). The largest difference between marginal/unsatisfactory and satisfactory PGY2s was trustworthiness (0.78). The largest differences between marginal/unsatisfactory and satisfactory PGY3s were ethical behavior (1.17), incorporating feedback (1.03), and professionalization (0.96). For PGY2s and PGY3s, no subcompetencies had outstanding discrimination. Conclusions Marginal/unsatisfactory pediatric residents had different subcompetency gaps at different training levels. While PGY1s may have global deficits, senior residents may have different performance deficiencies requiring individualized counseling and targeted performance improvement plans.
AB - Purpose To perform a derivation study to determine in which subcompetencies marginal/unsatisfactory pediatric residents had the greatest deficits compared with their satisfactorily performing peers and which subcompetencies best discriminated between marginal/unsatisfactory and satisfactorily performing residents. Method Multi-institutional cohort study of all 21 milestones (rated on four or five levels) reported to the Accreditation Council for Graduate Medical Education, and global marginal/unsatisfactory versus satisfactory performance reported to the American Board of Pediatrics. Data were gathered in 2013-2014. For each level of training (postgraduate year [PGY] 1, 2, and 3), mean differences between milestone levels of residents with marginal/unsatisfactory and satisfactory performance adjusted for clustering by program and C-statistics (area under receiver operating characteristic curve) were calculated. A Bonferroni-corrected significance threshold of.0007963 was used to account for multiple comparisons. Results Milestone and overall performance evaluations for 1,704 pediatric residents in 41 programs were obtained. For PGY1s, two subcompetencies had almost a one-point difference in milestone levels between marginal/unsatisfactory and satisfactory trainees and outstanding discrimination (≥ 0.90): organize/prioritize (0.93; C-statistic: 0.91) and transfer of care (0.97; C-statistic: 0.90). The largest difference between marginal/unsatisfactory and satisfactory PGY2s was trustworthiness (0.78). The largest differences between marginal/unsatisfactory and satisfactory PGY3s were ethical behavior (1.17), incorporating feedback (1.03), and professionalization (0.96). For PGY2s and PGY3s, no subcompetencies had outstanding discrimination. Conclusions Marginal/unsatisfactory pediatric residents had different subcompetency gaps at different training levels. While PGY1s may have global deficits, senior residents may have different performance deficiencies requiring individualized counseling and targeted performance improvement plans.
KW - Academic Performance
KW - Clinical Competence
KW - Cohort Studies
KW - Female
KW - Humans
KW - Internship and Residency
KW - Male
KW - Pediatrics/education
KW - Reproducibility of Results
KW - United States
UR - http://www.scopus.com/inward/record.url?scp=85021084270&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85021084270&partnerID=8YFLogxK
U2 - 10.1097/ACM.0000000000001775
DO - 10.1097/ACM.0000000000001775
M3 - Article
C2 - 28640031
AN - SCOPUS:85021084270
SN - 1040-2446
VL - 93
SP - 119
EP - 129
JO - Journal of Medical Education
JF - Journal of Medical Education
IS - 1
ER -