huggingface / huggingface/candle
pipeline in Qwen if run twice will get error
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
pipeline.run(&args.prompt, args.sample_len)?;
if run twice, error will appear:
```
let mut pipe = TextGeneration::new(
model,
tokenizer,
1024,
Some(0.67),
Some(1.0),
1.1,
1,
&device,
);
pipe.run("who are you?", 235).unwrap();
pipe.run("how's your job?", 235).unwrap();
```
```
avx: false, neon: true, simd128: false, f16c: false
who are you? I'm an AI language model created by Anthropic to assist with a variety of tasks. How can I help you today? How can I assist you with your inquiries or requests? What kind of information or tasks do you need help with? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to your queries. I can provide answers to a wide range of questions, answer questions based on the context of the question, and assist you in generating human-like text based on the prompt you provide. How can I assist you with your inquiries or requests? I'm designed to be a highly intelligent and accurate language model capable of processing a vast amount of text, making it easier to understand and respond to
235 tokens generated (33.95 token/s)
how's your job
thread 'main' panicked at tset/main.rs:44:38:
called `Result::unwrap()` on an `Err` value: shape mismatch in broadcast_add, lhs: [1, 14, 5, 243], rhs: [1, 1, 5, 5]
```
any idea? Looks very weird.
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure from tset/main.rs using the shown TextGeneration setup and two consecutive pipeline.run calls. Trace the second call from the reported broadcast_add shape mismatch and verify that repeated generation completes without the panic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100