tensorflow / tensorflow/privacy
Intuition behind evaluating attack performance on data not in the training set
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 477
- Avg merge
- 22h 12m
- Merged PRs (30d)
- 1
Description
It seems like when creating the attack data, those in the training set are set as 0 and not in the training set are set as 1:
https://github.com/tensorflow/privacy/blob/caf71c11bcdf3f6301db6916a37901b22798c992/tensorflow_privacy/privacy/membership_inference_attack/models.py#L78-L79
https://github.com/tensorflow/privacy/blob/caf71c11bcdf3f6301db6916a37901b22798c992/tensorflow_privacy/privacy/membership_inference_attack/membership_inference_attack.py#L85-L88
In this way, the attack performance (auc and advantage) is evaluated on the data that are not in the training set instead of data in the training set, as pos_label for tpr and fpr in metrics.roc_curve is 1 by default.
I feel like most lieterature evaluates attack performance on the data in the training set instead as people want to see how well an adversary can distinguish if a sample is in the training set. Is there a specific reason for evaluating on the data that are not in the training set here?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked sections of tensorflow_privacy/privacy/membership_inference_attack/models.py and membership_inference_attack.py, then trace how labels reach metrics.roc_curve and its default pos_label. Done means establishing whether the label convention intentionally evaluates non-members or whether the attack metrics should represent membership in the training set.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100