[etLLM] Runtime components to support LLM models
Open
Nobody has claimed this yet.
module: llm
triaged
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🚀 The feature, motivation and pitch
Design solutions to handle challenges of an LLM runtime (on top of ExecuTorch core):
- Different tokenizer needs
- Increasing number of modalities
- Components or APIs for app-level needs, like reasoning and agentic logics
Sub issues can be created for each of the runtime components.
Alternatives
No response
Additional context
There’s the LLM extension to provide fundamental components running LLM models via ExecuTorch, with example runners for Llama and Llava models.
RFC (Optional)
No response
cc @mergennachin @cccclai @helunwencser @dvorjackz
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.