google-research / google-research/DP-FTRL

FTRL is identical to SGD for unconstrained problem when no noise is added

Open
#1 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
38
Forks
7
PR merge metrics
No merged PRs in 30d

Description

On line https://github.com/google-research/DP-FTRL/blob/main/optimizers.py#L59,

Why is `ms + (-gs - nz) / alpha`? This makes FTRL not FTRL is identical to SGD if learning rate is not 1.0. Shouldn't it be
`ms + (-gs - nz) * alpha`?

Contributor guide

Open the contributing guide

Research direction

Start in optimizers.py at line 59 and inspect how the FTRL update uses ms, gs, nz, and alpha. Compare the current division by alpha with the proposed multiplication for an unconstrained problem without noise and a learning rate other than 1.0. Done means the intended FTRL behavior is established and the issue is resolved or clarified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.