Automated Validation of Crowdsourced Data

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Ensuring the accuracy of information or data contributed by the crowd is amongst the challenges in crowdsourcing initiatives. Data that do not meet certain criteria set by the crowdsourcer are also submitted by the crowd in a crowdsourcing initiative due to its openness. Thus, there is a need to ensure that only valid data are being captured before the data are being processed further. However, manually validating the data is not practical due to the high volume of data involved in crowdsourcing and their unpredictable nature. Therefore, in this research, an automated algorithm to validate crowdsourced data was developed. The objective was to identify the processes needed to enable the validation of crowdsourced data to be performed automatically. Two types of validation were included; task validation and worker validation. Kuder-Richardson Formula 20 was used to compute validity of task and mean formula converted to percentage was used in computing worker validity. The algorithm was implemented by embedding it in a prototype crowdsourcing application called AsnafCircle that crowdsourced information on eligible asnaf (alms recipient) from the public. Evaluation showed that the algorithm was able to automatically compute values that determine task and worker validity. Evaluation by experts also conformed the necessity of the processes that constitute the algorithm. The presence of this algorithm will help to ensure validity of contributed data in crowdsourcing initiatives, hence, improving their reliability.

Original languageEnglish
Title of host publication2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538691755
DOIs
Publication statusPublished - 02 Jul 2018
Event16th IEEE Student Conference on Research and Development, SCOReD 2018 - Selangor, Malaysia
Duration: 26 Nov 201828 Nov 2018

Publication series

Name2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018

Conference

Conference16th IEEE Student Conference on Research and Development, SCOReD 2018
CountryMalaysia
CitySelangor
Period26/11/1828/11/18

Fingerprint

evaluation
embedding
prototypes

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Signal Processing
  • Electrical and Electronic Engineering
  • Instrumentation

Cite this

Ibrahim, Z., Aris, H., & Mansur, A. (2018). Automated Validation of Crowdsourced Data. In 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018 [8711108] (2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SCORED.2018.8711108
Ibrahim, Zailani ; Aris, Hazleen ; Mansur, Aishah. / Automated Validation of Crowdsourced Data. 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018. Institute of Electrical and Electronics Engineers Inc., 2018. (2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018).
@inproceedings{becf2ef4d7ce43529f3b2277b13971c5,
title = "Automated Validation of Crowdsourced Data",
abstract = "Ensuring the accuracy of information or data contributed by the crowd is amongst the challenges in crowdsourcing initiatives. Data that do not meet certain criteria set by the crowdsourcer are also submitted by the crowd in a crowdsourcing initiative due to its openness. Thus, there is a need to ensure that only valid data are being captured before the data are being processed further. However, manually validating the data is not practical due to the high volume of data involved in crowdsourcing and their unpredictable nature. Therefore, in this research, an automated algorithm to validate crowdsourced data was developed. The objective was to identify the processes needed to enable the validation of crowdsourced data to be performed automatically. Two types of validation were included; task validation and worker validation. Kuder-Richardson Formula 20 was used to compute validity of task and mean formula converted to percentage was used in computing worker validity. The algorithm was implemented by embedding it in a prototype crowdsourcing application called AsnafCircle that crowdsourced information on eligible asnaf (alms recipient) from the public. Evaluation showed that the algorithm was able to automatically compute values that determine task and worker validity. Evaluation by experts also conformed the necessity of the processes that constitute the algorithm. The presence of this algorithm will help to ensure validity of contributed data in crowdsourcing initiatives, hence, improving their reliability.",
author = "Zailani Ibrahim and Hazleen Aris and Aishah Mansur",
year = "2018",
month = "7",
day = "2",
doi = "10.1109/SCORED.2018.8711108",
language = "English",
series = "2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018",
address = "United States",

}

Ibrahim, Z, Aris, H & Mansur, A 2018, Automated Validation of Crowdsourced Data. in 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018., 8711108, 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018, Institute of Electrical and Electronics Engineers Inc., 16th IEEE Student Conference on Research and Development, SCOReD 2018, Selangor, Malaysia, 26/11/18. https://doi.org/10.1109/SCORED.2018.8711108

Automated Validation of Crowdsourced Data. / Ibrahim, Zailani; Aris, Hazleen; Mansur, Aishah.

2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018. Institute of Electrical and Electronics Engineers Inc., 2018. 8711108 (2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Automated Validation of Crowdsourced Data

AU - Ibrahim, Zailani

AU - Aris, Hazleen

AU - Mansur, Aishah

PY - 2018/7/2

Y1 - 2018/7/2

N2 - Ensuring the accuracy of information or data contributed by the crowd is amongst the challenges in crowdsourcing initiatives. Data that do not meet certain criteria set by the crowdsourcer are also submitted by the crowd in a crowdsourcing initiative due to its openness. Thus, there is a need to ensure that only valid data are being captured before the data are being processed further. However, manually validating the data is not practical due to the high volume of data involved in crowdsourcing and their unpredictable nature. Therefore, in this research, an automated algorithm to validate crowdsourced data was developed. The objective was to identify the processes needed to enable the validation of crowdsourced data to be performed automatically. Two types of validation were included; task validation and worker validation. Kuder-Richardson Formula 20 was used to compute validity of task and mean formula converted to percentage was used in computing worker validity. The algorithm was implemented by embedding it in a prototype crowdsourcing application called AsnafCircle that crowdsourced information on eligible asnaf (alms recipient) from the public. Evaluation showed that the algorithm was able to automatically compute values that determine task and worker validity. Evaluation by experts also conformed the necessity of the processes that constitute the algorithm. The presence of this algorithm will help to ensure validity of contributed data in crowdsourcing initiatives, hence, improving their reliability.

AB - Ensuring the accuracy of information or data contributed by the crowd is amongst the challenges in crowdsourcing initiatives. Data that do not meet certain criteria set by the crowdsourcer are also submitted by the crowd in a crowdsourcing initiative due to its openness. Thus, there is a need to ensure that only valid data are being captured before the data are being processed further. However, manually validating the data is not practical due to the high volume of data involved in crowdsourcing and their unpredictable nature. Therefore, in this research, an automated algorithm to validate crowdsourced data was developed. The objective was to identify the processes needed to enable the validation of crowdsourced data to be performed automatically. Two types of validation were included; task validation and worker validation. Kuder-Richardson Formula 20 was used to compute validity of task and mean formula converted to percentage was used in computing worker validity. The algorithm was implemented by embedding it in a prototype crowdsourcing application called AsnafCircle that crowdsourced information on eligible asnaf (alms recipient) from the public. Evaluation showed that the algorithm was able to automatically compute values that determine task and worker validity. Evaluation by experts also conformed the necessity of the processes that constitute the algorithm. The presence of this algorithm will help to ensure validity of contributed data in crowdsourcing initiatives, hence, improving their reliability.

UR - http://www.scopus.com/inward/record.url?scp=85066490883&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066490883&partnerID=8YFLogxK

U2 - 10.1109/SCORED.2018.8711108

DO - 10.1109/SCORED.2018.8711108

M3 - Conference contribution

T3 - 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018

BT - 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Ibrahim Z, Aris H, Mansur A. Automated Validation of Crowdsourced Data. In 2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018. Institute of Electrical and Electronics Engineers Inc. 2018. 8711108. (2018 IEEE 16th Student Conference on Research and Development, SCOReD 2018). https://doi.org/10.1109/SCORED.2018.8711108