huggingface / huggingface/swift-transformers
Allow discrete sequence lengths (enumerated input shapes)
- Dominant language
- Swift
- Stars
- 1.4k
- Forks
- 209
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 4
Description
When the Core ML model is converted using flexible shapes, it doesn't run on GPU or ANE. We are currently using a fixed sequence length, but a better solution (to explore) would be to:
- Use [`EnumeratedShapes`](https://apple.github.io/coremltools/source/coremltools.converters.mil.input_types.html#enumeratedshapes) to allow several possible input lengths. This change can be tested by converting manually using [`coremltools`](https://github.com/apple/coremltools), and then adapting the code in [`exporters`](https://github.com/huggingface/exporters).
- Update the generation code to select the shortest sequence length from the enumerated set that is larger than the current sequence.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.