huggingface / huggingface/transformers.js

[Feature request] onnx model compression

Open
#284 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

Thanks for releasing models with reduced size https://twitter.com/xenovacom/status/1698742891118493905 .

I was thinking of further reduction using compression algorithm like brotli. I have tested current whisper-base.en (~51MB) can be reduced to ~26MB using:

```sh
brotli decoder_model_merged_quantized.onnx -o decoder_model_merged_quantized.onnx.br -Z -f
```

However huggingface_hub doesnt seem to be capable ATM https://github.com/huggingface/huggingface_hub/issues/1446 .

So my idea / feature request is, whether instead of compressing on flight, it would be possible to:

1. compress in advance and commit compressed *.onnx.br files together with the original *.onnx ones
2. in runtime/JS check if brotli is supported by browser, and if so request *.onnx.br

In order for this to be transparent `fetch()` for browser, such .br files should be served with extra http headers:

```
Header set Content-Encoding br
Header append Vary Accept-Encoding
```

And preferably additional information on the about the original filesize (so js `fetch()` can recognise and report progress properly) i.e.

```
Header set x-file-size 123456789
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.