Researchers propose a new approach to embedding text for collective decision-making that prioritizes preferential similarity over semantic similarity. The method uses synthetic training data to separate preference signals (stance and values) from semantic nuisance (style and wording), improving preference prediction across deliberation datasets.
This research addresses a fundamental limitation in applying AI to democratic processes. When participants express opinions as free-form text, standard embeddings rely on semantic similarity—how words and concepts relate linguistically. However, collective decision-making requires preferential similarity: how well a person agrees with expressed views. The critical insight is that these two measures correlate by accident, not design, masking failures when correlation breaks down.
The invariance problem the authors identify is that embedding models encode both preference-relevant information (political stance, values) and irrelevant semantic noise (writing style, vocabulary choice). Off-the-shelf embeddings optimize for semantic tasks, inadvertently capturing some preference signals through accidental correlation. This creates false confidence in preference prediction when the underlying geometry actually reflects writing style rather than genuine agreement.
The proposed solution uses synthetic training data specifically designed to break the semantic-preference correlation. By forcing the model to predict preferences independent of stylistic variation, the optimal scoring function shifts away from cosine similarity toward genuinely preference-aligned representations. Testing across 11 online deliberation datasets demonstrates measurable improvement in preference prediction accuracy.
For the AI industry, this work has implications for participatory governance platforms, civic tech applications, and any system requiring preference aggregation from natural language. The methodology could enhance recommendation systems that must distinguish between semantic relevance and user preference alignment. As AI increasingly mediates collective decision-making, embedding preferences accurately becomes essential to maintaining trust in algorithmic fairness and representation.
- →Standard text embeddings measure semantic similarity, not the preferential similarity needed for fair collective decision-making.
- →Semantic and preferential similarity accidentally correlate, masking embedding failures in preference prediction.
- →Synthetic training data designed to break semantic-preference correlation significantly improves preference prediction accuracy.
- →The invariance problem affects any AI system using embeddings for preference aggregation or consensus-building.
- →This research enables more accurate algorithmic support for participatory governance and online deliberation platforms.