Confusion about the prompt size
Open
- Dominant language
- Rust
- Stars
- 619
- Forks
- 74
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to play with rwkv's long context capabilities. As far as I understand, the linked https://huggingface.co/cgisky/RWKV-x070-Ai00/tree/main/world_v3/3b model should be enough for the RWKV capabilities of really long prompts, right? When I cross a few thousand characters (using the oapi endpoint and the test python script from this repo), I start getting silly responses. This happens even after bumping up max_tokens.
Am I misunderstanding what's possible to do here? Or doing something wrong?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.