Why is the current sharedRmsprop thread safe?
- Dominant language
- Lua
- Stars
- 263
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I've read the discussion per #15 and #50, but I still don't understand why the current sharedRmsprop impl avoids thread racing? Actually, the code still occasionally outputs NaN on my machine unless I set the thread be one. By tracking the error I can tell it is due to these two lines:
```lua
state.g:mul(momentum):addcmul(1 - momentum, dfdx, dfdx)
state.tmp:copy(state.g):add(epsilon):sqrt()
```
as `state.tmp` can become zero while being divided in the rest code and I guess the zeros are due to `state.tmp:copy(state.g)` from other thread where `state.g` happens to include 0s...
Meanwhile, by changing them to
```lua
state.g:mul(momentum):addcmul(1 - momentum, dfdx, dfdx)
state.tmp:sqrt(torch.add(state.g, epsilon))
```
the error seems to disappear.
It my modification reasonable? Or I just have to update OpenBLAS or something?
Contributor guide
Research direction
Start with the sharedRmsprop implementation and the two state.g/state.tmp expressions shown in the issue. Investigate whether concurrent updates can expose zero values or NaNs, and compare the proposed torch.add form under multiple threads. Done means determining whether the change is a valid fix or whether the issue comes from OpenBLAS or another threading interaction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100