huggingface / huggingface/sharp-transformers
It is hoped that feature processing of the whisper model can be supported in the future:)
- Dominant language
- C#
- Stars
- 200
- Forks
- 17
- PR merge metrics
- No merged PRs in 30d
Description
In the whisper model, it is exported to three onnx, namely encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx. The input dimension of encoder_model.onnx is fbank feature data of [1, 80, 300] dimension, but the data to extract fbank features is written in huggingface in python, so I hope that all about whisper can be used in future versions. Here are some personal information Links that I found useful :)
Looking forward to sharp-transformers getting better and better
Fbank feature extraction
https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/feature_extraction_whisper.py#L84
https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/feature_extraction_whisper.py#L250
https://github.com/huggingface/transformers/blob/main/src/transformers/audio_utils.py#L119
WhisperToken
https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/tokenization_whisper.py#L52
https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/tokenization_whisper.py#L744
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.