idSCD: Identifying Training Datasets through Semantic Correlation Descriptors

Researchers have developed a new method called Semantic Correlation Descriptors (SCDs) to identify whether a specific dataset was used to train a machine learning model by analyzing the spurious correlations embedded in its learned structure. This white-box approach outperforms existing black-box membership inference techniques, achieving up to 60% higher accuracy in detecting dataset membership across natural language and medical text classification tasks.

The research addresses a fundamental challenge in machine learning privacy and model transparency: determining which datasets contributed to a model's training. Traditional membership inference attacks rely on indirect signals like confidence scores or generated samples, but this work takes a fundamentally different approach by examining the semantic correlations that datasets uniquely imprint on models during training.

The significance of this discovery extends beyond academic interest. As machine learning models become increasingly central to critical applications—from healthcare to finance—understanding their training composition matters greatly. Models absorb dataset-specific quirks and spurious correlations that aren't causal but are predictive within that particular data distribution. By formalizing these patterns into Semantic Correlation Descriptors, the researchers created a fingerprinting mechanism that works without access to training procedures or model internals beyond the learned weights.

For the AI and machine learning community, this technique has dual implications. On one hand, it strengthens privacy concerns around model training data, suggesting that datasets leave detectable traces regardless of how carefully models are constructed. On the other hand, it provides researchers and practitioners with a diagnostic tool to verify model composition, audit training procedures, and detect unauthorized data usage—valuable for both defensive and investigative purposes.

The experimental validation across three distinct domains demonstrates robustness, though the method shows clear limitations when datasets share significant semantic overlap. The 60% relative performance improvement over existing baselines suggests this approach represents a meaningful advancement in membership inference methodology. Future work will likely focus on understanding these limitations and developing defenses against SCD-based detection.

• →SCDs identify dataset membership by analyzing semantic correlation structures learned during model training, outperforming existing black-box methods by up to 60% in ROC-AUC
• →The approach works as a white-box fingerprinting method requiring only the model's weights and target dataset, without needing leave-one-dataset-out reference models
• →Datasets leave dataset-specific traces through spurious correlations that models internalize, making them detectable signatures of training composition
• →Performance degrades when datasets share significant semantic similarity, indicating the method works best when training data sources have distinct particularities
• →This discovery strengthens privacy concerns while simultaneously providing tools for auditing model training procedures and detecting unauthorized data usage