Support Kyutai-labs Moshi and Hibiki models

🚀 The feature, motivation and pitch

As titled, support Moshi and Hibiki models via ExecuTorch on iPhone and Android.

Kyutai-labs is the OSS leading team on audio-in, audio-out LLM models. The sizes are suitable for on-device use cases to show duplex user-system speech dialogue.

For multimodality, there are vision models (llava and llama3.2 vision) supported by ExecuTorch. Moshi would be a great speech LLM model to be supported.

Alternatives

No response

Additional context

From Zermelo Fraenkel:

• Moshi and Hibiki have exactly the same architecture, it's made of mimi (the audio tokenizer) + a llama like 1B transformer for a hibiki. The main performance bottleneck is on the transformer, I think we can try running mimi on the cpu to starts with.
• Hibiki only supports French to English for now, we hope to have English to French soon. Nothing is language specific in the model so scaling to more languages is more a matter of time and compute.
• You can try out the iphone version here https://github.com/kyutai-labs/moshi-swift , it's very brittle though 🙂 Code for the nvidia/macbook version here: https://github.com/kyutai-labs/hibiki
• Some attempt to get an even smaller version to run on android is here: https://github.com/LaurentMazare/moshi-android (300M transformer, audio-in only so that it's less compute intense).

RFC (Optional)

Suggested process:

Leverage the existing llama_transformer, with a set of ExecuTorch infra built around it, like export_llama(https://github.com/pytorch/executorch/blob/main/examples/models/llama/export_llama.py) for export, quantization and lowering to backends.
The Moshi/hibiki logic and be implemented in its own model.py, in a specific model folder. A reference example is llava model, with some glue logic to connect different components.

cc @mergennachin @cccclai @helunwencser @dvorjackz

Start by reading examples/models/llama/llama_transformer.py and export_llama.py, then compare the component glue in examples/models/llava with the Moshi, Hibiki, and mimi architectures. Done means ExecuTorch can export, quantize, lower, and run the models on iPhone and Android.