The critical role of direct observation in entrustment decisions

Background: Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objectiv...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Matthew Sibbald, Muqtasid Mansoor, Michael Tsang, Sarah Blissett, Geoffrey Norman
Formato: article
Lenguaje:EN
Publicado: Canadian Medical Education Journal 2021
Materias:
Acceso en línea:https://doaj.org/article/78b98b3a37d341388d5f9bcbe5f982d3
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
Descripción
Sumario:Background: Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objective structured clinical exam (OSCE). Methods: Sixteen faculty examiners provided 287 retrospective and real-time entrustment ratings of 16 cardiology trainees during OSCE stations in 2019 and 2020. Reliability and validity of these ratings were assessed by comparing correlations across stations as a measure of reliability, differences across postgraduate years as an index of construct validity, correlation to standardized in-training exam (ITE) as a measure of criterion validity, and reclassification of entrustment as a measure of consequential validity. Results: Both retrospective and real-time assessments were highly reliable (all intra-class correlations >0.86). Both increased with year of postgraduate training. Real-time entrustment ratings were significantly correlated with standardized ITE scores; retrospective ratings were not. Real-time ratings explained 37% (2019) and 46% (2020) of variance in examination scores vs. 21% (2019) and 7% (2020) for retrospective ratings. Direct observation resulted in a different level of entrustment compared with retrospective ratings in 44% of cases (p = <0.001). Conclusions: Ratings based on direct observation made unique contributions to entrustment decisions.