tensorflow / tensorflow/privacy

Missing semi-supervised training components for PATE

Open
#164 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2k
Forks
477
Avg merge
22h 12m
Merged PRs (30d)
1

Description

Hello,
I am currently trying to reproduce the results for training PATE on SVHN for "Scalabe Private Learning with PATE" (Papernot et al.) with the code given in privacy/research/pate_2018.
I succed up to the point of student training, with teachers having a comparable accuracy and generating a comparable amount of labels with similar privacy budget.

The issues I face come up when trying to applying the semi-supervised training step described by the authors.
That is, the GAN based approache in their Papernot et al. 2017 and Virtual Adversarial Training (VAT) for the current paper.
For both approaches, an implementation seems to be missing in the given code for PATE.
Without semi-supervised learning, my results for a model trained on SVHN with parameters given by the 2018 paper show a clear gap to the presented results (around 6% acc at least).

I tried to implement a custom train-loop for semi-supervised learning with VAT in TensorFlow 2.4, following the paper on VAT Miyato et al..
Sadly, the but results are worse than without using it and simply training on a very small train dataset.

My question is: Can you point me to an implementation of the GAN/VAT based semi-supervised training method that was used to produce the results?
Or could you otherwise give me details on parameters / implementation details that are needed to make the semi-supervised training work with PATE?
That would be great, thanks a lot in advance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the student-training path in privacy/research/pate_2018 and compare it with the semi-supervised GAN and VAT methods described in the linked Papernot et al. papers. The custom TensorFlow 2.4 training loop is additional reference material; done would require an implementation or concrete parameters that reproduces the reported SVHN results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.