pytorch / pytorch/executorch

Selective operator build workflow for multiple models in a single runtime environment

Open
#18,104 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🚀 The feature, motivation and pitch

ExecuTorch provides several mechanisms for selective operator compilation (e.g. EXECUTORCH_SELECT_OPS_MODEL, EXECUTORCH_SELECT_OPS_YAML). These workflows appear to be primarily designed around building a runtime for a single model.

In deployments where a single runtime must support multiple models, each model may require a different set of operators. Currently this typically requires manually extracting and merging operator sets from multiple models before building.

It would be able to provide multiple models or YAML files as input for operator extraction, or to document a recommended workflow for building a runtime that supports multiple models.

This would simplify common scenarios where one ExecuTorch runtime needs to execute several models.

Alternatives

Have been using a solution by merging multiple YAML files outside of Executorch, so that is a usable workaround.

Additional context

No response

RFC (Optional)

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing EXECUTORCH_SELECT_OPS_MODEL and EXECUTORCH_SELECT_OPS_YAML workflows and how operator sets are extracted. Compare their current single-model behavior with the reported external YAML-merging workaround; done should mean either multiple models or YAML files are supported directly, or a documented workflow clearly explains how to build one runtime for several models.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.