Optimizing the length of computerized adaptive testing for the Force Concept Inventory

As a method to shorten the test time of the Force Concept Inventory (FCI), we suggest the use of computerized adaptive testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. In so doing,...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jun-ichiro Yasuda, Naohiro Mae, Michael M. Hull, Masa-aki Taniguchi
Formato:	article
Lenguaje:	EN
Publicado:	American Physical Society 2021
Materias:	Special aspects of education LC8-6691 Physics QC1-999
Acceso en línea:	https://doaj.org/article/9d63b410e9fb4eeeb6bd3964d49cb68a
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:9d63b410e9fb4eeeb6bd3964d49cb68a
record_format	dspace
spelling	oai:doaj.org-article:9d63b410e9fb4eeeb6bd3964d49cb68a2021-12-02T17:09:41ZOptimizing the length of computerized adaptive testing for the Force Concept Inventory10.1103/PhysRevPhysEducRes.17.0101152469-9896https://doaj.org/article/9d63b410e9fb4eeeb6bd3964d49cb68a2021-03-01T00:00:00Zhttp://doi.org/10.1103/PhysRevPhysEducRes.17.010115http://doi.org/10.1103/PhysRevPhysEducRes.17.010115https://doaj.org/toc/2469-9896As a method to shorten the test time of the Force Concept Inventory (FCI), we suggest the use of computerized adaptive testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. In so doing, the test length can be significantly shortened. As a step to develop a CAT-based version of the FCI (FCI-CAT), we examined the optimal test length of the FCI-CAT such that accuracy and precision [which we measure in terms of bias, standard error, and root-mean-square error (RMSE)] of Cohen’s d would be comparable to that of the full FCI for a given class size. First, we estimated the item parameters of the FCI items based on the three-parameter logistic model of item response theory, which are used in the algorithm of CAT. For this estimation, we used 2882 responses of Japanese university students. Second, we conducted a Monte Carlo simulation to analyze how the bias, standard error, and RMSE of Cohen’s d depend upon the test length. Third, we conducted a post hoc simulation to examine the consistency of the Monte Carlo results with what would have been obtained using empirical responses. For this comparison, we used 86 pairs of pre- and post- test responses of Japanese university students. As a result, we found that for a class size of 40, we may reduce the test length of the FCI-CAT to 15–19 items, thereby reducing the test time of the FCI to 50%–63%, with an accompanying decrease in accuracy and precision of only 5%–10%. The results of the Monte Carlo study and the post hoc simulation were consistent, which supports the adequacy of our Monte Carlo study and its relevance in terms of administering the FCI-CAT in real classrooms.Jun-ichiro YasudaNaohiro MaeMichael M. HullMasa-aki TaniguchiAmerican Physical SocietyarticleSpecial aspects of educationLC8-6691PhysicsQC1-999ENPhysical Review Physics Education Research, Vol 17, Iss 1, p 010115 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Special aspects of education LC8-6691 Physics QC1-999
spellingShingle	Special aspects of education LC8-6691 Physics QC1-999 Jun-ichiro Yasuda Naohiro Mae Michael M. Hull Masa-aki Taniguchi Optimizing the length of computerized adaptive testing for the Force Concept Inventory
description	As a method to shorten the test time of the Force Concept Inventory (FCI), we suggest the use of computerized adaptive testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. In so doing, the test length can be significantly shortened. As a step to develop a CAT-based version of the FCI (FCI-CAT), we examined the optimal test length of the FCI-CAT such that accuracy and precision [which we measure in terms of bias, standard error, and root-mean-square error (RMSE)] of Cohen’s d would be comparable to that of the full FCI for a given class size. First, we estimated the item parameters of the FCI items based on the three-parameter logistic model of item response theory, which are used in the algorithm of CAT. For this estimation, we used 2882 responses of Japanese university students. Second, we conducted a Monte Carlo simulation to analyze how the bias, standard error, and RMSE of Cohen’s d depend upon the test length. Third, we conducted a post hoc simulation to examine the consistency of the Monte Carlo results with what would have been obtained using empirical responses. For this comparison, we used 86 pairs of pre- and post- test responses of Japanese university students. As a result, we found that for a class size of 40, we may reduce the test length of the FCI-CAT to 15–19 items, thereby reducing the test time of the FCI to 50%–63%, with an accompanying decrease in accuracy and precision of only 5%–10%. The results of the Monte Carlo study and the post hoc simulation were consistent, which supports the adequacy of our Monte Carlo study and its relevance in terms of administering the FCI-CAT in real classrooms.
format	article
author	Jun-ichiro Yasuda Naohiro Mae Michael M. Hull Masa-aki Taniguchi
author_facet	Jun-ichiro Yasuda Naohiro Mae Michael M. Hull Masa-aki Taniguchi
author_sort	Jun-ichiro Yasuda
title	Optimizing the length of computerized adaptive testing for the Force Concept Inventory
title_short	Optimizing the length of computerized adaptive testing for the Force Concept Inventory
title_full	Optimizing the length of computerized adaptive testing for the Force Concept Inventory
title_fullStr	Optimizing the length of computerized adaptive testing for the Force Concept Inventory
title_full_unstemmed	Optimizing the length of computerized adaptive testing for the Force Concept Inventory
title_sort	optimizing the length of computerized adaptive testing for the force concept inventory
publisher	American Physical Society
publishDate	2021
url	https://doaj.org/article/9d63b410e9fb4eeeb6bd3964d49cb68a
work_keys_str_mv	AT junichiroyasuda optimizingthelengthofcomputerizedadaptivetestingfortheforceconceptinventory AT naohiromae optimizingthelengthofcomputerizedadaptivetestingfortheforceconceptinventory AT michaelmhull optimizingthelengthofcomputerizedadaptivetestingfortheforceconceptinventory AT masaakitaniguchi optimizingthelengthofcomputerizedadaptivetestingfortheforceconceptinventory
_version_	1718381514002006016

Optimizing the length of computerized adaptive testing for the Force Concept Inventory

Ejemplares similares