Establish reproducible Criterion benchmark baseline on main

Objective

Establish a small, reproducible Criterion benchmark baseline on main before the CubeCL HAL migration changes execution behavior.

The historical benchmark branch currently has no commits ahead of main, so there is no code left to migrate. Implement the benchmark harness directly against the current public API.

Scope

Add Criterion configuration and representative host/Rayon benchmarks for:

1. array construction or allocation;
2. one unary elementwise operation;
3. one binary operation, including a supported broadcast case;
4. one reduction;
5. matrix multiplication;
6. one non-contiguous view, indexing, or transform operation.

Use a small shape matrix that includes:

• a latency-oriented small case;
• a representative medium case;
• a throughput-oriented large case that remains safe in the documented Tembo environment.

If a listed operation is not currently available through a stable public API, select the nearest representative public operation and document the substitution in the PR. Do not add product functionality merely to make a benchmark possible.

Measurement rules

• Use deterministic, seeded input generation.
• Benchmark the operation under test separately from input construction where practical.
• Include allocation or transfer costs only in benchmarks explicitly named as end-to-end.
• Use black_box or the current Criterion equivalent to prevent optimization from eliminating work.
• Reuse inputs only where doing so does not change semantics or measure an unintended cache.
• Record dtype, shape, feature set, thread count, Rust toolchain, OS, CPU model and logical/physical core counts.
• Keep benchmark IDs stable and machine-readable.
• Bound sample size, warmup and measurement duration so a complete local run has a documented finite budget.
• Do not compare results from different machines as though they were directly equivalent.

Procedure

1. Add a compatible Criterion development dependency and Cargo benchmark target configuration.
2. Create a shared deterministic fixture/input utility for the benchmark targets.
3. Implement the six benchmark categories and documented shape matrix.
4. Verify that each benchmark actually exercises the intended ha-ndarray operation.
5. Add a CI step that compiles benchmarks with cargo bench --no-run --no-default-features; do not run timing-sensitive benchmarks on ordinary shared CI.
6. Run the suite once in the Tembo environment and retain the Criterion report or machine-readable output as a PR artifact.
7. Document local execution, environment recording, result comparison and how future CubeCL/OpenCL benchmark groups should reuse the baseline.
8. Remove or correct stale documentation that instructs contributors to migrate code from the benchmark branch.

Acceptance criteria

• Criterion is configured without changing the public ha-ndarray API.
• All six benchmark categories exist or have an explicitly justified public-API substitution.
• Inputs are deterministic and setup is not accidentally included in operation-only measurements.
• Small, medium and bounded large cases are documented.
• Benchmark names encode the operation, dtype and shape sufficiently for comparison.
• cargo bench --no-run --no-default-features passes in a clean checkout.
• The full host-only benchmark suite completes within the documented Tembo budget.
• Existing host-only tests pass.
• The PR includes exact environment identity and an initial result artifact.
• Documentation explains how #34–#50 should add comparable CubeCL groups without changing these host baselines.
• No performance threshold, supported-hardware claim or backend-retirement decision is introduced.

Non-goals

• CubeCL implementation or evaluation
• OpenCL, GPU, or browser benchmarking
• NumPy or other external-library comparisons
• Declaring performance regressions or release thresholds
• Modifying array algorithms to improve the initial numbers
• Removing the historical branch
• Changing default features or the public API

Required evidence

The draft PR must include:

• changed-file list;
• benchmark inventory and chosen operation mapping;
• exact commands executed;
• environment/toolchain identity;
• benchmark compilation result;
• initial Criterion report or machine-readable result artifact;
• total benchmark duration;
• test results;
• known limitations and any unavailable all-feature/OpenCL validation.

A faster or slower number is not, by itself, evidence of correctness.

Tembo execution contract

automation:
  eligible: true
  executor_profile: tembo
  readiness: ready
  primary_repository: TinyChain-Inc/ha-ndarray
  base_branch: main
  writable_scope:
    - Cargo.toml
    - Cargo.lock
    - benches/**
    - tests/benchmark_support/**
    - .github/workflows/**
    - README.md
    - CONTRIBUTING.md
    - docs/benchmarking/**
  prerequisites: []
  concurrency_group: ha-ndarray-benchmark-baseline
  required_commands:
    - "cargo fmt --all -- --check"
    - "cargo test --no-default-features"
    - "cargo bench --no-run --no-default-features"
    - "cargo bench --no-default-features"
  evidence:
    deterministic_inputs: required
    benchmark_inventory: required
    initial_result_artifact: required
    exact_environment_identity: required
    total_duration: required
    accelerator_evidence: not-required
  external_actions: none
  draft_pr_only: true
  completion_authority: human
  budget:
    size: medium
    max_revisions: 3
    max_wall_minutes: 120

Program relationship

This benchmark baseline supports the CubeCL roadmap in #33, the minimal HAL slice in #34, the reproducible accelerator lane in #50, and the protected OpenCL-retirement decision in #40. It does not depend on those issues and may begin immediately.

Start with Cargo.toml and the permitted paths: benches/, tests/benchmark_support/, .github/workflows/, README.md, CONTRIBUTING.md, and docs/benchmarking/. Run the required formatting, host-only tests, and benchmark compilation commands first, then map six public API operations to deterministic fixtures and documented shapes. Done means the bounded host suite runs, CI compiles it without timing runs, and the PR includes the required environment, commands, results, and limitations.