ACM paper warns: synthetic data “anonymity” claims don’t survive real privacy attacks
Daily Brief2 min read

ACM paper warns: synthetic data “anonymity” claims don’t survive real privacy attacks

An ACM paper argues that synthetic datasets and the models that generate them can still leak personal information under privacy attacks. The authors recom…

daily-briefsynthetic-dataprivacy-attacksdifferential-privacymembership-inferencedata-governance

A new ACM paper argues that “synthetic = anonymous” is an unsafe default: both the synthetic dataset and the generator model can leak personal information under standard privacy attacks. The authors push for model-centric anonymity assessments and conclude differential privacy is more robust than similarity-based metrics.

Rethinking Anonymity Claims in Synthetic Data Generation

Researchers are challenging a common operational shortcut in synthetic data programs: treating synthetic outputs as inherently anonymous. In the ACM paper Rethinking Anonymity Claims in Synthetic Data Generation, the authors argue that synthetic datasets—and the models used to generate them—can still expose personal information when subjected to privacy attacks designed to infer whether a person was in the training data or to re-identify individuals.

The paper recommends shifting from dataset-only checks to model-centric anonymity assessments, evaluating risk at two layers: (1) the released synthetic records and (2) the generator itself (which may be queried, shared, or deployed in ways that enable inference). The authors also find differential privacy to be more robust than similarity-based privacy metrics, which can provide a false sense of safety if they don’t reflect attacker capabilities.

  • “Looks different” isn’t a privacy guarantee. Similarity-based metrics can miss membership-inference and identifiability risk; teams should validate against explicit attack models, not just record-level resemblance scores.
  • Assess the generator, not just the dataset. If the model is exposed (via API, internal access, or vendor tooling), it becomes a privacy surface; governance needs to cover query controls, logging, and red-teaming of the model itself.
  • DP is positioned as the safer baseline. If you need a defensible privacy story for regulators, auditors, or customers, the paper’s conclusion strengthens the case for DP mechanisms over ad hoc “anonymity” claims.
  • Update release checklists. Practical next steps include membership-inference testing, identifiability evaluations, and documenting threat models for both synthetic outputs and any deployed generator.