huggingface / huggingface/transformers
BatchFeature should cast to `np.float32` by default
Open
@patrickvonplaten is already working on this.
Since Jul 23, 2021.
WIP
- Dominant language
- Python
- Stars
- 166k
- Forks
- 34.6k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 281
Description
Currently the default dtype for Speech Feature Extractors is numpy.float64 which leads to two problems:
- It makes the data processing extremely expensive for the RAM. Many sound formats are stored in int16 (such as
.wav) and are then transformed to float64 which unnecessarly increases RAM by a factor of 4. We should at least stick tofloat32 - Currently we have added some hacks to the Wav2Vec2 and Speech2TextTransformer feature extractors to prevent Double vs. Float dtype mismatches: https://github.com/huggingface/transformers/blob/f6e254474cb4f90f8a168a599b9aaf3544c37890/src/transformers/models/wav2vec2/feature_extraction_wav2vec2.py#L87
The main problem is that np.asarray([....]) by default creates a np.float64 array and that we just pass that format along.
=> We should either always cast to float32 in BatchFeature (see here: https://github.com/huggingface/transformers/blob/f6e254474cb4f90f8a168a599b9aaf3544c37890/src/transformers/feature_extraction_utils.py#L151) or add a flag dtype to BatchFeature.
@patrickvonplaten
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.