huggingface / huggingface/transformers.js
[Feature request] onnx model compression
- Dominant language
- JavaScript
- Stars
- 16.3k
- Forks
- 1.2k
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 6
Description
Thanks for releasing models with reduced size https://twitter.com/xenovacom/status/1698742891118493905 .
I was thinking of further reduction using compression algorithm like brotli. I have tested current whisper-base.en (~51MB) can be reduced to ~26MB using:
```sh
brotli decoder_model_merged_quantized.onnx -o decoder_model_merged_quantized.onnx.br -Z -f
```
However huggingface_hub doesnt seem to be capable ATM https://github.com/huggingface/huggingface_hub/issues/1446 .
So my idea / feature request is, whether instead of compressing on flight, it would be possible to:
1. compress in advance and commit compressed *.onnx.br files together with the original *.onnx ones
2. in runtime/JS check if brotli is supported by browser, and if so request *.onnx.br
In order for this to be transparent `fetch()` for browser, such .br files should be served with extra http headers:
```
Header set Content-Encoding br
Header append Vary Accept-Encoding
```
And preferably additional information on the about the original filesize (so js `fetch()` can recognise and report progress properly) i.e.
```
Header set x-file-size 123456789
```
Contributor guide
Assessment
This issue has not been assessed yet.