Consensus voting is a process to ensure a higher quality of the annotated batches. In the context of validation workflows, consensus consists in one of the four types of workflows to evaluate the correctness of a labeled data.
This process consists in labeling the same piece of data multiple times and validating the data only if all annotators agree on the label. That is, the data will only be used if all the labelers agree on the label of the data.
In this case, at least two of our annotators label each piece of information. In case of disagreement, we will need to review the asset.
The four types of workflow to evaluate the correctness of a labeled data and, therefore, measure the quality of the annotation are: without validation, with review, consensus voting and honeypot.
Interested in better understanding how to evaluate quality in a data labeling project? Take a look at our article!
Synonyms : Ground-truth
