tensorflow / tensorflow/privacy

implimenting DP-GAN in Keras

Open
#128 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2k
Forks
477
Avg merge
22h 12m
Merged PRs (30d)
1

Description

Hi. I'm using TFP and Keras to implement DP-GAN. I define the optimizer as follows:

optimizer = DPRMSPropGaussianOptimizer(l2_norm_clip=args.l2_norm_clip,
                                               noise_multiplier=args.noise_multiplier,
                                               num_microbatches=args.microbatches,
                                               learning_rate=args.discriminator_lr,
                                               decay=args.discriminator_decay)
loss = tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.losses.Reduction.NONE)
model.compile(loss=loss, optimizer=optimizer, metrics=['accuracy'])

I also use model.train_on_batch in Keras to train both generator and discriminator. There is no need to apply noise on the gradients of discriminator when fake images are processed to the best of my knowledge. Therefore, I train the discriminator two times one for real and another for fake images. Accordingly, I change l2_clip_norm and noise_multiplier two times, while to make DP ineffective in applying discriminator gradients for fake images, I set optimizer's parameters using keras.backend as follows:

tf.keras.backend.set_value(model.optimizer.l2_norm_clip, 20.0)
tf.keras.backend.set_value(model.optimizer.noise_multiplier, 0.0)
tf.keras.backend.set_value(model.optimizer.num_microbatches, 1)

I have no problem running the code, and I know TFP has issues in Keras. However, to be sure, I want a second opinion. So, In case you notice any problems or have any suggestions, I'll be grateful to share it with me.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Keras train_on_batch flow and DPRMSPropGaussianOptimizer configuration shown in the issue. Check whether changing l2_norm_clip, noise_multiplier, and num_microbatches between real and fake batches preserves the intended privacy accounting; done requires a maintainer-confirmed, documented approach or a reproducible correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.