Weight Stripping
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
Enable creating engines (currently, MXR files, eventually perhaps dynamic objects) without embedding the weights in the engine.
Use cases:
(1) Support compilation for various batch sizes without duplicating the weights.
(2) Support multiple execution configurations with different quantization options (including mixed precision), without necessarily having to embed the weights in all the created engines.
(3) multi-GPU execution may benefit from this also, especially when it comes to creating multiple multiGPU execution configurations (partitions, execution schedules)
Technical considerations:
How do we treat literals?
Perhaps we need to have the MXR files contain the steps required to recreate the literals from the weights' file, and that may require a new type ( finalized lliterals vs future literal or meta-literal)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.