vllm-project / vllm-project/aibrix
Do LLM Cache Support V100 hardware?
Open
area/kv-cache
kind/support
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 98
Description
I using V100 gpu to testing deploy Distributed KV Cache exmaple, unfortunately it's failed, because requires flash attention backend.
Contributor guide
Research direction
Start by reproducing the Distributed KV Cache example on V100 hardware and inspect the flash attention backend requirement shown in the report. Determine whether V100 is supported; done means either the example works on V100 or the compatibility limitation and required hardware are clearly documented.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100