Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments

Introduction There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how t...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Caroline R. Paul, Michael S. Ryan, Gary L. Beck Dallaghan, Thanakorn Jirasevijinda, Patricia D. Quigley, Janice L. Hanson, Amal M. Khidir, Jean Petershack, Joseph Jackson, Linda Tewksbury, Mary Esther M. Rocha
Formato:	article
Lenguaje:	EN
Publicado:	Association of American Medical Colleges 2019
Materias:	Evaluation Editor's Choice Assessment Curriculum Development Validity Evidence Quantitative Research Medicine (General) R5-920 Education L
Acceso en línea:	https://doaj.org/article/fb37938bf6a7466aaa0d9a887cf294b9
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:fb37938bf6a7466aaa0d9a887cf294b9
record_format	dspace
spelling	oai:doaj.org-article:fb37938bf6a7466aaa0d9a887cf294b92021-11-22T13:42:27ZCollecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments10.15766/mep_2374-8265.108172374-8265https://doaj.org/article/fb37938bf6a7466aaa0d9a887cf294b92019-04-01T00:00:00Zhttp://www.mededportal.org/doi/10.15766/mep_2374-8265.10817https://doaj.org/toc/2374-8265Introduction There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. Methods The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. Results The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. Discussion This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes.Caroline R. PaulMichael S. RyanGary L. Beck DallaghanThanakorn JirasevijindaPatricia D. QuigleyJanice L. HansonAmal M. KhidirJean PetershackJoseph JacksonLinda TewksburyMary Esther M. RochaAssociation of American Medical CollegesarticleEvaluationEditor's ChoiceAssessmentCurriculum DevelopmentValidity EvidenceQuantitative ResearchMedicine (General)R5-920EducationLENMedEdPORTAL, Vol 15 (2019)
institution	DOAJ
collection	DOAJ
language	EN
topic	Evaluation Editor's Choice Assessment Curriculum Development Validity Evidence Quantitative Research Medicine (General) R5-920 Education L
spellingShingle	Evaluation Editor's Choice Assessment Curriculum Development Validity Evidence Quantitative Research Medicine (General) R5-920 Education L Caroline R. Paul Michael S. Ryan Gary L. Beck Dallaghan Thanakorn Jirasevijinda Patricia D. Quigley Janice L. Hanson Amal M. Khidir Jean Petershack Joseph Jackson Linda Tewksbury Mary Esther M. Rocha Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
description	Introduction There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. Methods The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. Results The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. Discussion This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes.
format	article
author	Caroline R. Paul Michael S. Ryan Gary L. Beck Dallaghan Thanakorn Jirasevijinda Patricia D. Quigley Janice L. Hanson Amal M. Khidir Jean Petershack Joseph Jackson Linda Tewksbury Mary Esther M. Rocha
author_facet	Caroline R. Paul Michael S. Ryan Gary L. Beck Dallaghan Thanakorn Jirasevijinda Patricia D. Quigley Janice L. Hanson Amal M. Khidir Jean Petershack Joseph Jackson Linda Tewksbury Mary Esther M. Rocha
author_sort	Caroline R. Paul
title	Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
title_short	Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
title_full	Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
title_fullStr	Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
title_full_unstemmed	Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
title_sort	collecting validity evidence: a hands-on workshop for medical education assessment instruments
publisher	Association of American Medical Colleges
publishDate	2019
url	https://doaj.org/article/fb37938bf6a7466aaa0d9a887cf294b9
work_keys_str_mv	AT carolinerpaul collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT michaelsryan collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT garylbeckdallaghan collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT thanakornjirasevijinda collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT patriciadquigley collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT janicelhanson collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT amalmkhidir collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT jeanpetershack collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT josephjackson collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT lindatewksbury collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT maryesthermrocha collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments
_version_	1718417592633262080

Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments

Ejemplares similares