Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries

Researchers demonstrate that transformer-based tabular foundation models leak sensitive information through their attention mechanisms, enabling effective membership inference attacks despite being pre-trained on synthetic data. The study proposes both an attack method (AMIA) and a defense strategy inspired by k-anonymity that reduces privacy leakage by 50% while maintaining model performance.

This research exposes a critical privacy vulnerability in machine learning systems that are increasingly deployed for sensitive applications. Tabular foundation models have gained adoption because they're typically trained on synthetic data, creating a false sense of security among practitioners. However, the paper reveals that the real risk emerges during inference when sensitive records are provided as context examples—a standard practice in few-shot learning scenarios. The attention mechanism, fundamental to transformer architectures, inadvertently reveals whether specific data points were included in the training set through concentration patterns in its weights.

The attention-based membership inference attack (AMIA) achieves a 7.7% performance improvement over traditional confidence-based attacks, particularly in low false-positive regimes where security matters most. This matters because it fundamentally challenges assumptions about model safety in production environments. The authors demonstrate that fine-tuning amplifies this risk, as samples with increased prediction confidence after fine-tuning become more vulnerable to attacks. This creates a paradox: improving model accuracy through fine-tuning inadvertently increases privacy exposure.

For practitioners deploying these models, the proposed k-anonymity-inspired defense offers practical mitigation without requiring model retraining or accuracy sacrifices. The approach reduces AMIA leakage by 50% while maintaining 96.1% of predictive utility, making it implementable in production systems. However, the research highlights that attention mechanisms themselves may be inherently privacy-leaky, suggesting deeper architectural changes may ultimately be necessary. Organizations handling sensitive tabular data should now evaluate their inference procedures and consider implementing similar protections, particularly when processing high-risk queries identifiable through AMIA scoring.

• →Transformer attention mechanisms leak sufficient information to enable membership inference attacks on sensitive data in context examples
• →AMIA outperforms classical confidence-based attacks by 7.7% on average, especially when minimizing false positives
• →A k-anonymity-inspired defense reduces privacy leakage by 50% with only 3.9% performance degradation
• →Fine-tuning amplifies privacy risks by increasing memorization signals in samples with confidence improvements
• →Inference-time defenses can protect high-risk queries without model retraining or noise injection