A reinforcement learning model to inform optimal decision paths for HIV elimination

The 'Ending the HIV Epidemic (EHE)' national plan aims to reduce annual HIV incidence in the United States from 38,000 in 2015 to 9300 by 2025 and 3300 by 2030. Diagnosis and treatment are two most effective interventions, and thus, identifying corresponding optimal combinations of testing...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Seyedeh N. Khatami, Chaitra Gopalappa
Formato:	article
Lenguaje:	EN
Publicado:	AIMS Press 2021
Materias:	decision-making in epidemics agent-based simulation modeling national hiv goals reinforcement learning ending the hiv epidemic artificial intelligence in public health Biotechnology TP248.13-248.65 Mathematics QA1-939
Acceso en línea:	https://doaj.org/article/f18f1efdd24747e58680c9ce413feb68
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f18f1efdd24747e58680c9ce413feb68
record_format	dspace
spelling	oai:doaj.org-article:f18f1efdd24747e58680c9ce413feb682021-11-23T02:31:14ZA reinforcement learning model to inform optimal decision paths for HIV elimination10.3934/mbe.20213801551-0018https://doaj.org/article/f18f1efdd24747e58680c9ce413feb682021-09-01T00:00:00Zhttps://www.aimspress.com/article/doi/10.3934/mbe.2021380?viewType=HTMLhttps://doaj.org/toc/1551-0018The 'Ending the HIV Epidemic (EHE)' national plan aims to reduce annual HIV incidence in the United States from 38,000 in 2015 to 9300 by 2025 and 3300 by 2030. Diagnosis and treatment are two most effective interventions, and thus, identifying corresponding optimal combinations of testing and retention-in-care rates would help inform implementation of relevant programs. Considering the dynamic and stochastic complexity of the disease and the time dynamics of decision-making, solving for optimal combinations using commonly used methods of parametric optimization or exhaustive evaluation of pre-selected options are infeasible. Reinforcement learning (RL), an artificial intelligence method, is ideal; however, training RL algorithms and ensuring convergence to optimality are computationally challenging for large-scale stochastic problems. We evaluate its feasibility in the context of the EHE goal. We trained an RL algorithm to identify a 'sequence' of combinations of HIV-testing and retention-in-care rates at 5-year intervals over 2015-2070 that optimally leads towards HIV elimination. We defined optimality as a sequence that maximizes quality-adjusted-life-years lived and minimizes HIV-testing and care-and-treatment costs. We show that solving for testing and retention-in-care rates through appropriate reformulation using proxy decision-metrics overcomes the computational challenges of RL. We used a stochastic agent-based simulation to train the RL algorithm. As there is variability in support-programs needed to address barriers to care-access, we evaluated the sensitivity of optimal decisions to three cost-functions. The model suggests to scale-up retention-in-care programs to achieve and maintain high annual retention-rates while initiating with a high testing-frequency but relaxing it over a 10-year period as incidence decreases. Results were mainly robust to the uncertainty in costs. However, testing and retention-in-care alone did not achieve the 2030 EHE targets, suggesting the need for additional interventions. The results from the model demonstrated convergence. RL is suitable for evaluating phased public health decisions for infectious disease control.Seyedeh N. KhatamiChaitra GopalappaAIMS Pressarticledecision-making in epidemicsagent-based simulation modelingnational hiv goalsreinforcement learningending the hiv epidemicartificial intelligence in public healthBiotechnologyTP248.13-248.65MathematicsQA1-939ENMathematical Biosciences and Engineering, Vol 18, Iss 6, Pp 7666-7684 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	decision-making in epidemics agent-based simulation modeling national hiv goals reinforcement learning ending the hiv epidemic artificial intelligence in public health Biotechnology TP248.13-248.65 Mathematics QA1-939
spellingShingle	decision-making in epidemics agent-based simulation modeling national hiv goals reinforcement learning ending the hiv epidemic artificial intelligence in public health Biotechnology TP248.13-248.65 Mathematics QA1-939 Seyedeh N. Khatami Chaitra Gopalappa A reinforcement learning model to inform optimal decision paths for HIV elimination
description	The 'Ending the HIV Epidemic (EHE)' national plan aims to reduce annual HIV incidence in the United States from 38,000 in 2015 to 9300 by 2025 and 3300 by 2030. Diagnosis and treatment are two most effective interventions, and thus, identifying corresponding optimal combinations of testing and retention-in-care rates would help inform implementation of relevant programs. Considering the dynamic and stochastic complexity of the disease and the time dynamics of decision-making, solving for optimal combinations using commonly used methods of parametric optimization or exhaustive evaluation of pre-selected options are infeasible. Reinforcement learning (RL), an artificial intelligence method, is ideal; however, training RL algorithms and ensuring convergence to optimality are computationally challenging for large-scale stochastic problems. We evaluate its feasibility in the context of the EHE goal. We trained an RL algorithm to identify a 'sequence' of combinations of HIV-testing and retention-in-care rates at 5-year intervals over 2015-2070 that optimally leads towards HIV elimination. We defined optimality as a sequence that maximizes quality-adjusted-life-years lived and minimizes HIV-testing and care-and-treatment costs. We show that solving for testing and retention-in-care rates through appropriate reformulation using proxy decision-metrics overcomes the computational challenges of RL. We used a stochastic agent-based simulation to train the RL algorithm. As there is variability in support-programs needed to address barriers to care-access, we evaluated the sensitivity of optimal decisions to three cost-functions. The model suggests to scale-up retention-in-care programs to achieve and maintain high annual retention-rates while initiating with a high testing-frequency but relaxing it over a 10-year period as incidence decreases. Results were mainly robust to the uncertainty in costs. However, testing and retention-in-care alone did not achieve the 2030 EHE targets, suggesting the need for additional interventions. The results from the model demonstrated convergence. RL is suitable for evaluating phased public health decisions for infectious disease control.
format	article
author	Seyedeh N. Khatami Chaitra Gopalappa
author_facet	Seyedeh N. Khatami Chaitra Gopalappa
author_sort	Seyedeh N. Khatami
title	A reinforcement learning model to inform optimal decision paths for HIV elimination
title_short	A reinforcement learning model to inform optimal decision paths for HIV elimination
title_full	A reinforcement learning model to inform optimal decision paths for HIV elimination
title_fullStr	A reinforcement learning model to inform optimal decision paths for HIV elimination
title_full_unstemmed	A reinforcement learning model to inform optimal decision paths for HIV elimination
title_sort	reinforcement learning model to inform optimal decision paths for hiv elimination
publisher	AIMS Press
publishDate	2021
url	https://doaj.org/article/f18f1efdd24747e58680c9ce413feb68
work_keys_str_mv	AT seyedehnkhatami areinforcementlearningmodeltoinformoptimaldecisionpathsforhivelimination AT chaitragopalappa areinforcementlearningmodeltoinformoptimaldecisionpathsforhivelimination AT seyedehnkhatami reinforcementlearningmodeltoinformoptimaldecisionpathsforhivelimination AT chaitragopalappa reinforcementlearningmodeltoinformoptimaldecisionpathsforhivelimination
_version_	1718417413030019072

A reinforcement learning model to inform optimal decision paths for HIV elimination

Ejemplares similares