dotnet / dotnet/machinelearning
Review and document ONNX testing coverage
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
ONNX tests exist across transformer, integration, conversion, and sequence-type test projects. However, there is no current coverage matrix for supported scenarios, validation paths, platforms, and public APIs.
Issues such as #2500 and #3375 also show areas that need review.
### Goal
Review the current ONNX test suite, document its coverage, and create focused follow-up issues for confirmed gaps.
### Scope
Review coverage for:
- Model loading from files, streams, and memory
- Fixed, dynamic, unknown, and variable-length dimensions
- Tensor, sequence, and map types
- Single and multiple inputs and outputs
- Input and output schema validation
- Image layout, channel order, and normalization
- Model save, load, disposal, and session options
- CPU and GPU execution and fallback
- Windows, Linux, and macOS
- Output correctness against expected ONNX Runtime results
- Use of current public APIs
### Acceptance criteria
- [ ] Create a matrix that maps each scenario to its tests and platforms.
- [ ] Review #2500 and #3375 and record their current status.
- [ ] Identify obsolete, duplicate, skipped, or platform-limited tests.
- [ ] Identify tests that do not validate output correctness.
- [ ] Create a focused follow-up issue for each confirmed gap, or document why no test is necessary.
Contributor guide
Research direction
Start by reviewing the ONNX tests in the transformer, integration, conversion, and sequence-type test projects, then examine issues #2500 and #3375. Build a matrix covering the listed scenarios and platforms, note obsolete, duplicate, skipped, limited, or output-unvalidated tests, and create focused follow-up issues for confirmed gaps or document why none are needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- documentation, machine-learning, testing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100