TinyChain-Inc / TinyChain-Inc/ha-ndarray
[Validation] Add reproducible CubeCL CPU and accelerator conformance runner
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Objective
Provide the reproducible native validation lane used to accept CubeCL HAL work on CPU and at least one designated accelerator, with exact environment identity and retained machine-readable evidence.
Scope
- Define the initial supported validation matrix after #34 records the CubeCL runtime decision.
- Pin runner image/toolchain, Rust, CubeCL, OS packages and test/benchmark commands.
- Record device vendor/model, driver, runtime/backend, memory and relevant capabilities.
- Run shared host-versus-CubeCL correctness fixtures and collect structured results.
- Separate cold compilation, warm execution, transfer and peak-memory measurements.
- Retain logs, results and benchmark artifacts by commit.
- Detect unavailable, changed or incorrectly emulated hardware and fail/skip explicitly.
- Keep benchmark thresholds and supported-platform declarations human-owned.
Acceptance criteria
- A clean run can reproduce the CPU lane from documented commands.
- A designated accelerator lane proves actual device dispatch, not compilation or CPU emulation.
- Every artifact identifies commit, toolchain, CubeCL, OS, device, driver and backend/runtime.
- Numerical failures include operation, dtype, shape, expected/actual summary and tolerance contract without dumping sensitive data.
- Cold/warm/transfer/memory metrics are separate and machine-readable.
- Missing or changed hardware cannot be reported as a passing accelerator test.
- An independent reviewer can download and inspect retained artifacts.
- Runner credentials cannot access production or unrelated repositories/resources.
Non-goals
Choosing final performance thresholds, declaring broad hardware support, browser/WASM validation (owned by #39), or retiring OpenCL.
Tembo execution contract
automation:
eligible: true
executor_profile: tembo
readiness: blocked
primary_repository: TinyChain-Inc/ha-ndarray
base_branch: main
writable_scope:
- ".github/workflows/**"
- "scripts/validation/**"
- "tests/conformance/**"
- "benches/**"
- "docs/validation/**"
prerequisites:
- "#34 runtime/version decision"
concurrency_group: cubecl-validation-infrastructure
required_commands:
- "cargo fmt --all -- --check"
- "cargo test --no-default-features --features cubecl"
- "cargo test --all-features"
- "run the documented designated-accelerator validation command introduced by this issue"
evidence:
cpu_lane: required
actual_designated_accelerator: required
exact_environment_identity: required
structured_results: required
retained_artifacts: required
external_actions: designated-runner-only
draft_pr_only: true
completion_authority: human-infrastructure-and-HAL-review
budget: {size: large, max_revisions: 3, max_wall_minutes: 180}
Program: #33
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with prerequisite issue #34, then inspect .github/workflows/, scripts/validation/, tests/conformance/, benches/, and docs/validation/** to define the agreed runtime and validation matrix. Run the required Cargo checks and the documented accelerator command; done means reproducible CPU and actual-device results with environment identity, structured metrics, retained artifacts, and explicit handling of unavailable hardware.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, rust
- Domain
- ci-cd, infrastructure, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100