Aggregating Reliable Submissions in Crowdsourcing Systems

Crowdsourcing is a cost-effective method that gathers crowd wisdom to solve machine-hard problems. In crowdsourcing systems, requesters post tasks for obtaining reliable solutions. Nevertheless, since workers have various expertise and knowledge background, they probably deliver low-quality and ambi...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ayswarya R. Kurup, G. P. Sajeev, J. Swaminathan
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Answer aggregation expertness estimation probabilistic model quality control cost minimization Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/51b6538ea0dd427c878fa4ba81d86588
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:51b6538ea0dd427c878fa4ba81d86588
record_format	dspace
spelling	oai:doaj.org-article:51b6538ea0dd427c878fa4ba81d865882021-11-20T00:00:56ZAggregating Reliable Submissions in Crowdsourcing Systems2169-353610.1109/ACCESS.2021.3127994https://doaj.org/article/51b6538ea0dd427c878fa4ba81d865882021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9614193/https://doaj.org/toc/2169-3536Crowdsourcing is a cost-effective method that gathers crowd wisdom to solve machine-hard problems. In crowdsourcing systems, requesters post tasks for obtaining reliable solutions. Nevertheless, since workers have various expertise and knowledge background, they probably deliver low-quality and ambiguous submissions. A task aggregation scheme is generally employed in crowdsourcing systems, to deal with this problem. Existing methods mainly focus on structured submissions and also do not consider the cost incurred for completing a task. We exploit features of submissions to improve the task aggregation for proposing a method which is applicable to both structured and unstructured tasks. Moreover, existing probabilistic methods for answer aggregation are sensitive to sparsity. Our approach uses a generative probabilistic model that incorporates similarity in answers along with worker and task features. Thereafter, we present a method for minimizing the cost of tasks, that eventually leverages the quality of answers. We conduct experiments on empirical data that demonstrates the effectiveness of our method compared to state-of-the-art approaches.Ayswarya R. KurupG. P. SajeevJ. SwaminathanIEEEarticleAnswer aggregationexpertness estimationprobabilistic modelquality controlcost minimizationElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 153058-153071 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Answer aggregation expertness estimation probabilistic model quality control cost minimization Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Answer aggregation expertness estimation probabilistic model quality control cost minimization Electrical engineering. Electronics. Nuclear engineering TK1-9971 Ayswarya R. Kurup G. P. Sajeev J. Swaminathan Aggregating Reliable Submissions in Crowdsourcing Systems
description	Crowdsourcing is a cost-effective method that gathers crowd wisdom to solve machine-hard problems. In crowdsourcing systems, requesters post tasks for obtaining reliable solutions. Nevertheless, since workers have various expertise and knowledge background, they probably deliver low-quality and ambiguous submissions. A task aggregation scheme is generally employed in crowdsourcing systems, to deal with this problem. Existing methods mainly focus on structured submissions and also do not consider the cost incurred for completing a task. We exploit features of submissions to improve the task aggregation for proposing a method which is applicable to both structured and unstructured tasks. Moreover, existing probabilistic methods for answer aggregation are sensitive to sparsity. Our approach uses a generative probabilistic model that incorporates similarity in answers along with worker and task features. Thereafter, we present a method for minimizing the cost of tasks, that eventually leverages the quality of answers. We conduct experiments on empirical data that demonstrates the effectiveness of our method compared to state-of-the-art approaches.
format	article
author	Ayswarya R. Kurup G. P. Sajeev J. Swaminathan
author_facet	Ayswarya R. Kurup G. P. Sajeev J. Swaminathan
author_sort	Ayswarya R. Kurup
title	Aggregating Reliable Submissions in Crowdsourcing Systems
title_short	Aggregating Reliable Submissions in Crowdsourcing Systems
title_full	Aggregating Reliable Submissions in Crowdsourcing Systems
title_fullStr	Aggregating Reliable Submissions in Crowdsourcing Systems
title_full_unstemmed	Aggregating Reliable Submissions in Crowdsourcing Systems
title_sort	aggregating reliable submissions in crowdsourcing systems
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/51b6538ea0dd427c878fa4ba81d86588
work_keys_str_mv	AT ayswaryarkurup aggregatingreliablesubmissionsincrowdsourcingsystems AT gpsajeev aggregatingreliablesubmissionsincrowdsourcingsystems AT jswaminathan aggregatingreliablesubmissionsincrowdsourcingsystems
_version_	1718419831157424128

Aggregating Reliable Submissions in Crowdsourcing Systems

Ejemplares similares