huggingface / huggingface/candle
Short context length on Qwen quantized examples.
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Running any of the quantized examples so far they all seam to have a 1024 token limit.
```
cargo run --example quantized-qwen3 --release --features cuda,cudnn -- --which 4b --prompt "1802tokens later..."
```
```
Error: shape mismatch on target dim, dst: 1024, src: 1802 + 0
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the command in the quantized-qwen3 example with the 1802-token prompt. Trace how the example determines or passes the context length and compare it with the 1024-token shape reported in the error. Done means the quantized Qwen example accepts prompts longer than 1024 tokens without the shape mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100