Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews

Abstract Background AMSTAR-2 (‘A Measurement Tool to Assess Systematic Reviews, version 2’) and ROBIS (‘Risk of Bias in Systematic Reviews’) are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constru...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Mateusz J. Swierz, Dawid Storman, Joanna Zajac, Magdalena Koperny, Paulina Weglarz, Wojciech Staskiewicz, Magdalena Gorecka, Anna Skuza, Adam Wach, Klaudia Kaluzinska, Justyna Bochenek-Cibor, Bradley C. Johnston, Malgorzata M. Bala
Formato:	article
Lenguaje:	EN
Publicado:	BMC 2021
Materias:	AMSTAR-2 ROBIS Quality Systematic review Meta-analysis Nutrition Medicine (General) R5-920
Acceso en línea:	https://doaj.org/article/37958f4bd84c4fbdae0687fd3e0eff10
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:37958f4bd84c4fbdae0687fd3e0eff10
record_format	dspace
spelling	oai:doaj.org-article:37958f4bd84c4fbdae0687fd3e0eff102021-11-28T12:38:41ZSimilarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews10.1186/s12874-021-01457-w1471-2288https://doaj.org/article/37958f4bd84c4fbdae0687fd3e0eff102021-11-01T00:00:00Zhttps://doi.org/10.1186/s12874-021-01457-whttps://doaj.org/toc/1471-2288Abstract Background AMSTAR-2 (‘A Measurement Tool to Assess Systematic Reviews, version 2’) and ROBIS (‘Risk of Bias in Systematic Reviews’) are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition. Methods We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116). Results We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet’s agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers’ conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan. Conclusions Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users’ wishing to interpret the importance of SR/MA results.Mateusz J. SwierzDawid StormanJoanna ZajacMagdalena KopernyPaulina WeglarzWojciech StaskiewiczMagdalena GoreckaAnna SkuzaAdam WachKlaudia KaluzinskaJustyna Bochenek-CiborBradley C. JohnstonMalgorzata M. BalaBMCarticleAMSTAR-2ROBISQualitySystematic reviewMeta-analysisNutritionMedicine (General)R5-920ENBMC Medical Research Methodology, Vol 21, Iss 1, Pp 1-10 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	AMSTAR-2 ROBIS Quality Systematic review Meta-analysis Nutrition Medicine (General) R5-920
spellingShingle	AMSTAR-2 ROBIS Quality Systematic review Meta-analysis Nutrition Medicine (General) R5-920 Mateusz J. Swierz Dawid Storman Joanna Zajac Magdalena Koperny Paulina Weglarz Wojciech Staskiewicz Magdalena Gorecka Anna Skuza Adam Wach Klaudia Kaluzinska Justyna Bochenek-Cibor Bradley C. Johnston Malgorzata M. Bala Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
description	Abstract Background AMSTAR-2 (‘A Measurement Tool to Assess Systematic Reviews, version 2’) and ROBIS (‘Risk of Bias in Systematic Reviews’) are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition. Methods We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116). Results We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet’s agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers’ conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan. Conclusions Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users’ wishing to interpret the importance of SR/MA results.
format	article
author	Mateusz J. Swierz Dawid Storman Joanna Zajac Magdalena Koperny Paulina Weglarz Wojciech Staskiewicz Magdalena Gorecka Anna Skuza Adam Wach Klaudia Kaluzinska Justyna Bochenek-Cibor Bradley C. Johnston Malgorzata M. Bala
author_facet	Mateusz J. Swierz Dawid Storman Joanna Zajac Magdalena Koperny Paulina Weglarz Wojciech Staskiewicz Magdalena Gorecka Anna Skuza Adam Wach Klaudia Kaluzinska Justyna Bochenek-Cibor Bradley C. Johnston Malgorzata M. Bala
author_sort	Mateusz J. Swierz
title	Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
title_short	Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
title_full	Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
title_fullStr	Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
title_full_unstemmed	Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews
title_sort	similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using amstar-2 and robis: systematic survey of nutrition reviews
publisher	BMC
publishDate	2021
url	https://doaj.org/article/37958f4bd84c4fbdae0687fd3e0eff10
work_keys_str_mv	AT mateuszjswierz similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT dawidstorman similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT joannazajac similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT magdalenakoperny similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT paulinaweglarz similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT wojciechstaskiewicz similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT magdalenagorecka similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT annaskuza similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT adamwach similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT klaudiakaluzinska similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT justynabochenekcibor similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT bradleycjohnston similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews AT malgorzatambala similaritiesreliabilityandgapsinassessingthequalityofconductofsystematicreviewsusingamstar2androbissystematicsurveyofnutritionreviews
_version_	1718407826593808384

Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews

Ejemplares similares