AMD-AGI / AMD-AGI/AgentKernelArena
Standardize Holdout Tests with Comprehensive Shape Coverage
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 118
- Forks
- 14
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 8
Description
We should set up holdout tests for all existing and new tasks. These tests should cover commonly used input shapes, such as 128, 256, 512, 1024, 2048, 4096, and similar sizes.
If the current test coverage is insufficient, additional test cases should be added to ensure the evaluations are strict and comprehensive, providing strong guarantees for both correctness and performance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the existing task evaluations and any holdout-test infrastructure, then review how input shapes are currently covered. Add coverage for commonly used sizes such as 128, 256, 512, 1024, 2048, and 4096 across existing and new tasks. Done means the tests enforce correctness and performance expectations with comprehensive shape coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100