alibaba / alibaba/FederatedScope
WARNING: Skip the batch due to the loss is NaN, it may be caused by exceeding the precision or invalid labels
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 261
- PR merge metrics
- No merged PRs in 30d
Description
When I run
"python federatedscope/main.py --cfg federatedscope/llm/baseline/exp_yaml/dolly_lda/dolly_federate.yaml"
why there usually exists
"WARNING: Skip the batch due to the loss is NaN, it may be caused by exceeding the precision or invalid labels"?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running `python federatedscope/main.py --cfg federatedscope/llm/baseline/exp_yaml/dolly_lda/dolly_federate.yaml` and capture the batches that produce the warning. Trace where the warning is emitted, then inspect the configured labels and precision involved. Done means identifying the reproducible cause and documenting or correcting the conditions that trigger the NaN loss.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100