Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.

Recent advancements in in situ methods, such as multiplexed in situ RNA hybridization and in situ RNA sequencing, have deepened our understanding of the way biological processes are spatially organized in tissues. Automated image processing and spot-calling algorithms for analyzing in situ transcrip...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Jenny M Vo-Phamhi, Kevin A Yamauchi, Rafael Gómez-Sjöberg
Formato: article
Lenguaje:EN
Publicado: Public Library of Science (PLoS) 2021
Materias:
Acceso en línea:https://doaj.org/article/36be295d2ebd43fea6174c1e404119e8
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
id oai:doaj.org-article:36be295d2ebd43fea6174c1e404119e8
record_format dspace
spelling oai:doaj.org-article:36be295d2ebd43fea6174c1e404119e82021-12-02T19:58:07ZValidation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.1553-734X1553-735810.1371/journal.pcbi.1009274https://doaj.org/article/36be295d2ebd43fea6174c1e404119e82021-08-01T00:00:00Zhttps://doi.org/10.1371/journal.pcbi.1009274https://doaj.org/toc/1553-734Xhttps://doaj.org/toc/1553-7358Recent advancements in in situ methods, such as multiplexed in situ RNA hybridization and in situ RNA sequencing, have deepened our understanding of the way biological processes are spatially organized in tissues. Automated image processing and spot-calling algorithms for analyzing in situ transcriptomics images have many parameters which need to be tuned for optimal detection. Having ground truth datasets (images where there is very high confidence on the accuracy of the detected spots) is essential for evaluating these algorithms and tuning their parameters. We present a first-in-kind open-source toolkit and framework for in situ transcriptomics image analysis that incorporates crowdsourced annotations, alongside expert annotations, as a source of ground truth for the analysis of in situ transcriptomics images. The kit includes tools for preparing images for crowdsourcing annotation to optimize crowdsourced workers' ability to annotate these images reliably, performing quality control (QC) on worker annotations, extracting candidate parameters for spot-calling algorithms from sample images, tuning parameters for spot-calling algorithms, and evaluating spot-calling algorithms and worker performance. These tools are wrapped in a modular pipeline with a flexible structure that allows users to take advantage of crowdsourced annotations from any source of their choice. We tested the pipeline using real and synthetic in situ transcriptomics images and annotations from the Amazon Mechanical Turk system obtained via Quanti.us. Using real images from in situ experiments and simulated images produced by one of the tools in the kit, we studied worker sensitivity to spot characteristics and established rules for annotation QC. We explored and demonstrated the use of ground truth generated in this way for validating spot-calling algorithms and tuning their parameters, and confirmed that consensus crowdsourced annotations are a viable substitute for expert-generated ground truth for these purposes.Jenny M Vo-PhamhiKevin A YamauchiRafael Gómez-SjöbergPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Computational Biology, Vol 17, Iss 8, p e1009274 (2021)
institution DOAJ
collection DOAJ
language EN
topic Biology (General)
QH301-705.5
spellingShingle Biology (General)
QH301-705.5
Jenny M Vo-Phamhi
Kevin A Yamauchi
Rafael Gómez-Sjöberg
Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
description Recent advancements in in situ methods, such as multiplexed in situ RNA hybridization and in situ RNA sequencing, have deepened our understanding of the way biological processes are spatially organized in tissues. Automated image processing and spot-calling algorithms for analyzing in situ transcriptomics images have many parameters which need to be tuned for optimal detection. Having ground truth datasets (images where there is very high confidence on the accuracy of the detected spots) is essential for evaluating these algorithms and tuning their parameters. We present a first-in-kind open-source toolkit and framework for in situ transcriptomics image analysis that incorporates crowdsourced annotations, alongside expert annotations, as a source of ground truth for the analysis of in situ transcriptomics images. The kit includes tools for preparing images for crowdsourcing annotation to optimize crowdsourced workers' ability to annotate these images reliably, performing quality control (QC) on worker annotations, extracting candidate parameters for spot-calling algorithms from sample images, tuning parameters for spot-calling algorithms, and evaluating spot-calling algorithms and worker performance. These tools are wrapped in a modular pipeline with a flexible structure that allows users to take advantage of crowdsourced annotations from any source of their choice. We tested the pipeline using real and synthetic in situ transcriptomics images and annotations from the Amazon Mechanical Turk system obtained via Quanti.us. Using real images from in situ experiments and simulated images produced by one of the tools in the kit, we studied worker sensitivity to spot characteristics and established rules for annotation QC. We explored and demonstrated the use of ground truth generated in this way for validating spot-calling algorithms and tuning their parameters, and confirmed that consensus crowdsourced annotations are a viable substitute for expert-generated ground truth for these purposes.
format article
author Jenny M Vo-Phamhi
Kevin A Yamauchi
Rafael Gómez-Sjöberg
author_facet Jenny M Vo-Phamhi
Kevin A Yamauchi
Rafael Gómez-Sjöberg
author_sort Jenny M Vo-Phamhi
title Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
title_short Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
title_full Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
title_fullStr Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
title_full_unstemmed Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
title_sort validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.
publisher Public Library of Science (PLoS)
publishDate 2021
url https://doaj.org/article/36be295d2ebd43fea6174c1e404119e8
work_keys_str_mv AT jennymvophamhi validationandtuningofinsitutranscriptomicsimageprocessingworkflowswithcrowdsourcedannotations
AT kevinayamauchi validationandtuningofinsitutranscriptomicsimageprocessingworkflowswithcrowdsourcedannotations
AT rafaelgomezsjoberg validationandtuningofinsitutranscriptomicsimageprocessingworkflowswithcrowdsourcedannotations
_version_ 1718375769806209024