huggingface / huggingface/sentence-transformers
Tensor dtype
Open
- Dominant language
- Python
- Stars
- 19.1k
- Forks
- 2.9k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 61
Description
Hi,
Tensors in the saved `state_dict` have dtype `float32`. Is there a reason for that? I was able to cut the size of `pytorch_model.bin` roughly in half by converting tensors to float16 without loosing any accuracy. I'm using `distilbert-base-nli-mean-tokens`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing how tensors in the saved state_dict for distilbert-base-nli-mean-tokens are stored, then compare float32 and float16 versions for file size and accuracy. Done means the dtype choice is understood or an agreed change is implemented without loss of model accuracy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100