cumemallocator
Open
vllm
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
we should allocate things we intend to offload often(like optimizer, init policy) under cumemallocator similar to vllm. this way offload onload is fast
Contributor guide
Assessment
This issue has not been assessed yet.