dotnet / dotnet/machinelearning

Review and document ONNX testing coverage

Open
#7,687 1 comment 0 reactions 0 assignees View on GitHub
area-ONNX test-enhancement
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

ONNX tests exist across transformer, integration, conversion, and sequence-type test projects. However, there is no current coverage matrix for supported scenarios, validation paths, platforms, and public APIs.

Issues such as #2500 and #3375 also show areas that need review.

### Goal

Review the current ONNX test suite, document its coverage, and create focused follow-up issues for confirmed gaps.

### Scope

Review coverage for:

- Model loading from files, streams, and memory
- Fixed, dynamic, unknown, and variable-length dimensions
- Tensor, sequence, and map types
- Single and multiple inputs and outputs
- Input and output schema validation
- Image layout, channel order, and normalization
- Model save, load, disposal, and session options
- CPU and GPU execution and fallback
- Windows, Linux, and macOS
- Output correctness against expected ONNX Runtime results
- Use of current public APIs

### Acceptance criteria

- [ ] Create a matrix that maps each scenario to its tests and platforms.
- [ ] Review #2500 and #3375 and record their current status.
- [ ] Identify obsolete, duplicate, skipped, or platform-limited tests.
- [ ] Identify tests that do not validate output correctness.
- [ ] Create a focused follow-up issue for each confirmed gap, or document why no test is necessary.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ONNX tests in the transformer, integration, conversion, and sequence-type test projects, then examine issues #2500 and #3375. Build a matrix covering the listed scenarios and platforms, note obsolete, duplicate, skipped, limited, or output-unvalidated tests, and create focused follow-up issues for confirmed gaps or document why none are needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
documentation, machine-learning, testing
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.