pytorch / pytorch/executorch

[Request impl] Devtool end-to-end tests

Open
#9,778 18 comments 0 reactions 1 assignee View on GitHub

@HonestDeng is already working on this.

Since Apr 1, 2025.

good first issue module: devtools
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🚀 The feature, motivation and pitch

Devtools now have multiple components, like etdump, etrecrod, inspector, etc. Inpsector should take etdump and etrecord (optional) as input, but now we only have tests for each module separately, but do not have end-to-end tests, which may introduce some risk for our debugging pipeline. We would love for you to help us build end-to-end testing.

RFC

The overall picture of the e2e test will be like:

  1. Export a simple module (e.g. an module with only add or multiple operation) and dump into buffer; using etrecord to record information; see here for example but maybe use current export flow
  2. Using pybinding to load the buffer and equip with etdump
  3. Propagate the etrecord and etdump to inspector api ; choose one or two inspector functions (e.g. to_dataframe) and compare the result with your expectation.

This doc might be a good resource for you to understand the end2end pipeline better.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.