realtime quantization to workers
Open
enhancement
- Dominant language
- Rust
- Stars
- 3.1k
- Forks
- 208
- PR merge metrics
- No merged PRs in 30d
Description
in theory, if the master has the full res model, while streaming to workers it could quantize it automatically on the fly so that a worker can host a quantized version of the full model the master is using / orchestrating.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.