huggingface / huggingface/swift-transformers

Allow discrete sequence lengths (enumerated input shapes)

Open
#10 0 comments 5 reactions 0 assignees View on GitHub
modelling
Dominant language
Swift
Stars
1.4k
Forks
209
Avg merge
2d 7h
Merged PRs (30d)
4

Description

When the Core ML model is converted using flexible shapes, it doesn't run on GPU or ANE. We are currently using a fixed sequence length, but a better solution (to explore) would be to:

- Use [`EnumeratedShapes`](https://apple.github.io/coremltools/source/coremltools.converters.mil.input_types.html#enumeratedshapes) to allow several possible input lengths. This change can be tested by converting manually using [`coremltools`](https://github.com/apple/coremltools), and then adapting the code in [`exporters`](https://github.com/huggingface/exporters).
- Update the generation code to select the shortest sequence length from the enumerated set that is larger than the current sequence.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.