When using the --load-8bit parameter with Baichuan-13B-chat, an error occurs: None of the inputs have requires_grad=True. Gradients will be None.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
2023-08-31 14:49:16 | WARNING | transformers_modules.Baichuan-13B-Chat.modeling_baichuan | `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
2023-08-31 14:49:16 | ERROR | stderr | /data/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
2023-08-31 14:49:16 | ERROR | stderr | warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Baichuan-13B-chat case with the --load-8bit parameter and inspect the gradient-checkpointing warning from torch.utils.checkpoint. Determine why the inputs lack requires_grad=True, then verify that the warning no longer occurs in the affected configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100