The critical role of direct observation in entrustment decisions

Background: Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objectiv...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Matthew Sibbald, Muqtasid Mansoor, Michael Tsang, Sarah Blissett, Geoffrey Norman
Formato:	article
Lenguaje:	EN
Publicado:	Canadian Medical Education Journal 2021
Materias:	Education (General) L7-991 Medicine (General) R5-920
Acceso en línea:	https://doaj.org/article/78b98b3a37d341388d5f9bcbe5f982d3
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:78b98b3a37d341388d5f9bcbe5f982d3
record_format	dspace
spelling	oai:doaj.org-article:78b98b3a37d341388d5f9bcbe5f982d32021-12-01T22:35:20ZThe critical role of direct observation in entrustment decisions 10.36834/cmej.720401923-1202https://doaj.org/article/78b98b3a37d341388d5f9bcbe5f982d32021-11-01T00:00:00Zhttps://journalhosting.ucalgary.ca/index.php/cmej/article/view/72040https://doaj.org/toc/1923-1202 Background: Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objective structured clinical exam (OSCE). Methods: Sixteen faculty examiners provided 287 retrospective and real-time entrustment ratings of 16 cardiology trainees during OSCE stations in 2019 and 2020. Reliability and validity of these ratings were assessed by comparing correlations across stations as a measure of reliability, differences across postgraduate years as an index of construct validity, correlation to standardized in-training exam (ITE) as a measure of criterion validity, and reclassification of entrustment as a measure of consequential validity. Results: Both retrospective and real-time assessments were highly reliable (all intra-class correlations >0.86). Both increased with year of postgraduate training. Real-time entrustment ratings were significantly correlated with standardized ITE scores; retrospective ratings were not. Real-time ratings explained 37% (2019) and 46% (2020) of variance in examination scores vs. 21% (2019) and 7% (2020) for retrospective ratings. Direct observation resulted in a different level of entrustment compared with retrospective ratings in 44% of cases (p = <0.001). Conclusions: Ratings based on direct observation made unique contributions to entrustment decisions. Matthew SibbaldMuqtasid MansoorMichael TsangSarah BlissettGeoffrey NormanCanadian Medical Education JournalarticleEducation (General)L7-991Medicine (General)R5-920ENCanadian Medical Education Journal, Vol 12, Iss 5 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Education (General) L7-991 Medicine (General) R5-920
spellingShingle	Education (General) L7-991 Medicine (General) R5-920 Matthew Sibbald Muqtasid Mansoor Michael Tsang Sarah Blissett Geoffrey Norman The critical role of direct observation in entrustment decisions
description	Background: Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objective structured clinical exam (OSCE). Methods: Sixteen faculty examiners provided 287 retrospective and real-time entrustment ratings of 16 cardiology trainees during OSCE stations in 2019 and 2020. Reliability and validity of these ratings were assessed by comparing correlations across stations as a measure of reliability, differences across postgraduate years as an index of construct validity, correlation to standardized in-training exam (ITE) as a measure of criterion validity, and reclassification of entrustment as a measure of consequential validity. Results: Both retrospective and real-time assessments were highly reliable (all intra-class correlations >0.86). Both increased with year of postgraduate training. Real-time entrustment ratings were significantly correlated with standardized ITE scores; retrospective ratings were not. Real-time ratings explained 37% (2019) and 46% (2020) of variance in examination scores vs. 21% (2019) and 7% (2020) for retrospective ratings. Direct observation resulted in a different level of entrustment compared with retrospective ratings in 44% of cases (p = <0.001). Conclusions: Ratings based on direct observation made unique contributions to entrustment decisions.
format	article
author	Matthew Sibbald Muqtasid Mansoor Michael Tsang Sarah Blissett Geoffrey Norman
author_facet	Matthew Sibbald Muqtasid Mansoor Michael Tsang Sarah Blissett Geoffrey Norman
author_sort	Matthew Sibbald
title	The critical role of direct observation in entrustment decisions
title_short	The critical role of direct observation in entrustment decisions
title_full	The critical role of direct observation in entrustment decisions
title_fullStr	The critical role of direct observation in entrustment decisions
title_full_unstemmed	The critical role of direct observation in entrustment decisions
title_sort	critical role of direct observation in entrustment decisions
publisher	Canadian Medical Education Journal
publishDate	2021
url	https://doaj.org/article/78b98b3a37d341388d5f9bcbe5f982d3
work_keys_str_mv	AT matthewsibbald thecriticalroleofdirectobservationinentrustmentdecisions AT muqtasidmansoor thecriticalroleofdirectobservationinentrustmentdecisions AT michaeltsang thecriticalroleofdirectobservationinentrustmentdecisions AT sarahblissett thecriticalroleofdirectobservationinentrustmentdecisions AT geoffreynorman thecriticalroleofdirectobservationinentrustmentdecisions AT matthewsibbald criticalroleofdirectobservationinentrustmentdecisions AT muqtasidmansoor criticalroleofdirectobservationinentrustmentdecisions AT michaeltsang criticalroleofdirectobservationinentrustmentdecisions AT sarahblissett criticalroleofdirectobservationinentrustmentdecisions AT geoffreynorman criticalroleofdirectobservationinentrustmentdecisions
_version_	1718404107563171840

The critical role of direct observation in entrustment decisions

Ejemplares similares