google-research / google-research/DP-FTRL

accumulate noise in old trees without tree completion

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
38
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Hi,

Thanks for open-sourcing the code. I have a question on the code when we want to disable tree completion and restart the tree every k epochs. In particular, tree restarting is handled here (https://github.com/google-research/DP-FTRL/blob/513500a8e31e412972a7d457e9c66756e4a48348/main.py#L208-L215) where it would add the accumulated noise in the past old trees (including those in virtual nodes if using tree completion) to the sum of gradients.

If `tree_completion` is set to False, `self.record_last_noise` is False. Then when optimizer.restart() is called, the [assertion](https://github.com/google-research/DP-FTRL/blob/513500a8e31e412972a7d457e9c66756e4a48348/optimizers.py#L77) would fail. But we expect the algorithm to also add up the previous noise in this case, which it does not seem to do. Am I missing anything here?

Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.