pytorch / pytorch/executorch

[ARM] Ethos-U backend wheel or alternative versioned package (to avoid dev installs from source)

Open
#18,415 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: arm module: quantization partner: arm
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

šŸš€ The feature, motivation and pitch

I might be missing a workflow, but from what I understand today it’s hard to use the Arm Ethos‑U backend for QAT in a standard ML project environment without installing ExecuTorch + Arm requirements from source.

Concrete scenario (two python environments):

  1. .venv-model-authoring normal ML environment with:
    • custom model code
    • custom data-loaders
    • custom losses, optimizers
    • custom training/tuning loops
    • Managed with uv (could also be poetry/conda/pixi etc.)
  2. .venv-executorch-dev:
    • An environment with development ExecuTorch and Arm Ethos‑U backend installed from source. As instructed per documentation.

What works / what doesn’t:

  1. PTQ works with two envs: I can export from .venv-model-authoring a .pt2 exported-program, load it in the .venv-executorch-dev environment and feed the representative data through.
  2. QAT is a big struggle: seems like during the QAT loop I need pieces from both worlds - custom loaders, optimizers, losses from my authoring environment; and arm_quantizer from the Executorch ARM Ethos-U backend.

Why it’s difficult to ā€œjust install ExecuTorch from sourceā€ into .venv-model-authoring
Mixing a source/dev install into the main training environment tends to be fragile for reproducibility and dependency management:

  • the ML authoring project likely has a uv.lock file (or analogous), which is not respected by the source install of ExecuTorch and ARM backend.
  • the source install of ARM backend has hardcoded URLs (in contrast to configurable package indices) from where the dependencies are pulled, making it difficult to set up in firewall-restricted compute nodes.
  • source builds are generally slower than installing pre-packaged wheels in CI jobs.

Feature Request: A published package would be very helpful.

Maybe there is an alternative workflow that allows to serialize a model, ready for QAT tuning, that does not require arm_quantizer to be available in the model authoring environment?

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @kimishpatel @jerryzh168 @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the two-environment QAT workflow described in the issue, especially the need for arm_quantizer in the model-authoring environment. Determine whether a published Ethos-U backend wheel or a serialized QAT-ready model workflow can avoid source installs and hardcoded dependency URLs. Done means one supported workflow is defined for reproducible QAT use without installing ExecuTorch and the Arm backend from source.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, machine-learning, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.