tensorflow / tensorflow/privacy

Doubt about #steps in compute_dp_sgd_privacy.py

Open
#103 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2k
Forks
477
Avg merge
22h 12m
Merged PRs (30d)
1

Description

https://github.com/tensorflow/privacy/blob/319bf81bb166f4c43ca898a509bfdc56e1558280/tensorflow_privacy/privacy/analysis/compute_dp_sgd_privacy.py#L87

About steps = int(math.ceil(epochs * n / batch_size)), I have a doubt:

  • int seems useless, since ceil should return an integer, but that's not the subject.
  • shouldn't it be steps = int(epochs * math.ceil(n / batch_size)), since when n / batch_size is not an integer, one more (partial) iteration than math.floor(n / batch_size) is done during each epoch, isn't it ?
    On the other hand, even if I'm right, maybe the original formula is still correct, since it calculates a number of "whole steps equivalent" ??
    To illustrate what I mean: with a batch size of 256, 60000 entries in data set and 60 epochs, as 60000 / 256 = 234.375, the formula in code leads to 14063 steps (234.375 × 60 = 14062.5), when my reasoning leads to 235 × 60 = 14100.
    I hope I don't waste your time, but it seems worth verifying to me.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect tensorflow_privacy/privacy/analysis/compute_dp_sgd_privacy.py at line 87 and trace how the step count is used in the privacy calculation. Compare the formula with the intended handling of partial batches across epochs, using the reported 60,000-entry and batch-size-256 example. Done means confirming whether the calculation is correct or identifying the required correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.