huggingface / huggingface/nanotron
[BUG] Bug in Llama inference
Open
- Dominant language
- Python
- Stars
- 2.8k
- Forks
- 327
- Avg merge
- 36m
- Merged PRs (30d)
- 1
Description
- There's a bug of duplicate code w/ wrong indentation level when computing `attention_output` in `CausalSelfAttention._forward_inference`. Currently it's never computed.
https://github.com/huggingface/nanotron/blob/c737f00f01e65bc44e7624695351da7ed756ec31/src/nanotron/models/llama.py#L499-L666
- Wrong argument passed to `parametrizator_cls` when `init_model_randomly` for testing
https://github.com/huggingface/nanotron/blob/c737f00f01e65bc44e7624695351da7ed756ec31/src/nanotron/models/llama.py#L1095
Contributor guide
Assessment
This issue has not been assessed yet.