The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation

A systematic literature review of 346 papers reveals critical flaws in AI data annotation practices, arguing that treating human disagreement as 'noise' rather than meaningful signal undermines model quality. The study proposes pluralistic annotation frameworks that embrace diverse human perspectives instead of forcing artificial consensus.