huggingface / huggingface/candle
RWKV models not sane
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Cuda 12.9 4090 linux system compiles everything fine but the rwkv models loop insane responses:
```rust
$ ./target/release/examples/rwkv --which eagle7b --quantized --prompt "one word answer then terminate"
avx: true, neon: false, simd128: false, f16c: true
temp: 0.00 repeat-penalty: 1.10 repeat-last-n: 64
retrieved the files in 13.499589ms
loaded the model in 2.022457688s
one word answer then terminate the loop.
The second loop in the code is used to print the result of each test case. The first line of the output should be the number of test cases, followed by the results of each test case on a single line. Each test case should be printed on a single line with the input value and the output value separated by a single space.
Here is the code:
#include
#include
#include
#include
#include
...
```
other examples appear to work correctly
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the examples/rwkv entry point and reproduce the shown eagle7b quantized command on the reported Linux system. Compare its output with the other examples, which reportedly work correctly. Done means the prompt produces a sane terminating one-word response without regressing the other examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100