AI Membership Inference — Who Was in the Training Set?
Fennic's loan model was trained on a private list of past customers, and its public API returns a confidence score. Overfitted models are far more confident on records they were trained on — so the scores alone reveal who is in the private training set.
What is membership inference?
Membership inference asks one question: was this person's record in the data used to train the model? Models tend to be more confident about examples they were trained on than about new ones.
If an API returns exact confidence scores, an attacker can submit a record and compare the score with what they would expect for unseen data. Being on a training list can itself be sensitive, for example a list of loan applicants or patients.
A real case
We have not found a public breach from 2025 or 2026 attributed to membership inference, so we are not listing one. It is well established in research, and privacy regulators treat it as a reason why a model trained on personal data may itself count as personal data.
How to fix it
- Return a decision or a rounded score, not full-precision confidence.
- Reduce overfitting through regularisation and early stopping, since overfit models leak most.
- Consider differential privacy when training on sensitive records.
- Rate-limit and monitor accounts that submit many near-identical queries.
Questions people ask
Why does membership matter if no data is revealed?
Knowing that someone was in a dataset can reveal a fact about them, such as that they applied for a loan or were treated at a clinic.
Which models are most exposed?
Models that have memorised their training data, usually because the dataset was small or training ran too long.
What is differential privacy?
A training method that adds calibrated noise so the model's behaviour depends very little on any one record.
Read the full OWASP LLM Top 10 guide · All free labs · AI red team certification