Evaluate CubeCL as a replacement for OpenCL

OpenCL is deprecated by Apple and underperforms CUDA. At the same time, there are a number of new low-level Rust libraries for hardware acceleration of numerical computing:

• CubeCL (relatively simple but still efficient, allows operator/kernel fusion)
• XLA (powerful but significantly more complex)
• MLIR (future possibility, likely excessive for current goals)
• Candle ecosystem components (cudarc, metal-rs, etc.)
• tinygrad architecture (useful inspiration, not a direct dependency)
• Rusticl (does not address fusion/runtime planning requirements)

CubeCL Backend Migration

Implement a new CubeCL backend for ha-ndarray, retire the existing ocl/OpenCL backend after parity, and retain the host/Rayon backend as the fallback unless a separate measured decision changes that.

Why CubeCL: CubeCL provides a unified Rust-native execution model across CPU and GPU targets, with built-in support for runtime specialization, kernel fusion, and backend portability, allowing us to replace multiple backend-specific implementations with a single maintainable architecture.

Goals

• Add a CubeCL backend capable of executing all core ha-ndarray operations.
• Replace OpenCL-specific kernel management, compilation, device selection, and memory management currently implemented via ocl.
• Benchmark CubeCL CPU execution against the host/Rayon backend; replacement is explicitly outside this roadmap.
• Preserve and improve hardware acceleration support across available accelerator backends.
• Reduce overall backend complexity and maintenance burden.

Scope

• Introduce a backend abstraction layer independent of OpenCL.
• Implement CubeCL support for:
  • elementwise operations
  • reductions
  • matrix multiplication
  • type conversion
  • indexing and transforms
• Implement backend-independent:
  • kernel caching
  • device placement
  • memory accounting
  • backpressure and fallback behavior
• Support streaming execution for tensors larger than device memory (this can be implicit or explicit, preferably implicit in terms of automatically fusing sequences of operations at runtime and caching this fused kernel to avoid recompilation).
• Investigate runtime fusion using CubeCL-compatible fusion plans.

Browser Support Requirement

ha-ndarray must support browser deployment via:

ha-ndarray
    ↓
CubeCL
    ↓
WebGPU
    ↓
WASM
    ↓
Browser

This is a primary architectural goal of the migration.

Desired outcomes:

• Compile ha-ndarray to WebAssembly.
• Execute accelerated operations through CubeCL's WebGPU backend.
• Preserve the same ndarray API across native and browser environments.
• Reuse the same execution and fusion infrastructure on all supported targets.
• Avoid browser-specific forks of the execution engine.

Browser support should be considered a first-class use case rather than a future extension.

Long-Term Architecture

Array expression
    ↓
ha-ndarray execution IR
    ↓
fusion planning
    ↓
CubeCL backend
    ↓
CPU / CUDA / HIP / Metal / Vulkan

Migration Plan

1. Implement CubeCL backend alongside existing backends.
2. Achieve feature parity with the current OpenCL implementation.
3. Benchmark CubeCL CPU execution against existing rayon paths without making CPU-backend retirement a gate.
4. Deprecate the opencl feature once CubeCL reaches parity.
5. Remove ocl and OpenCL-specific infrastructure.
6. Keep host/Rayon as the supported fallback unless a separate benchmark-backed issue authorizes a change.

Success Criteria

• All existing ha-ndarray functionality runs on CubeCL.
• Hardware acceleration works on supported GPU backends.
• CPU execution remains available without accelerator hardware.
• Kernel compilation is cached and reused across executions.
• ocl and OpenCL-specific infrastructure can be retired without loss of functionality; host/Rayon remains available.

Tembo-executable HAL roadmap

Architecture and general parity

• #34 — HAL/runtime decision and minimal CubeCL vertical slice
• #41 — runtime, device, buffer, transfer, and constructors
• #42 — elementwise, broadcasting, and dtype conversion
• #43 — views, indexing, and transforms
• #35 — human acceptance rollup for #41–#43

Numerical parity

• #44 — reductions and numerical conformance
• #45 — matrix multiplication and linalg
• #36 — human acceptance rollup for #44–#45

Runtime fusion and streaming

• #46 — execution IR and materialization boundaries
• #47 — fusion legality, CubeCL lowering, and plan cache
• #37 — human acceptance rollup for #46–#47
• #48 — ordered block-stream contract and backpressure
• #49 — fused CubeCL block-stream execution
• #38 — human acceptance rollup for #48–#49

Platform and retirement validation

• #39 — actual-browser WASM/WebGPU conformance
• #50 — reproducible CPU/designated-accelerator validation runner
• #22 — benchmark migration
• #40 — protected parity/performance/release decision and OpenCL retirement

Dependency order

#34 → #41 ┬→ #42 ┬→ #44 ───────────────┐
          │      ├→ #45 ───────────────┤
          │      └→ #46 → #47 ─┬→ #49 ├→ #39 ─┐
          └→ #43 ───────────────┘      │       │
                 #48 ──────────────────┘       ├→ #40
                         #50 ──────────────────┤
                         #22 ──────────────────┘

#42 and #43 may proceed concurrently after #41. #44 and #45 may then proceed concurrently. #46 consumes general-operation semantics; #47 consumes #46. #48 may begin after the buffer foundation and proceeds independently of fusion; #49 joins #47 and #48.

Validation authority

• Host/Rayon execution remains the independent correctness oracle during migration.
• Fused execution must always be comparable with an unfused path.
• CPU execution does not prove accelerator support.
• Native WebGPU does not prove browser/WASM support.
• Every accelerator or browser result records exact hardware, driver/runtime, OS, Rust, CubeCL, browser, and artifact revisions.
• Tembo may implement, test, benchmark, and prepare draft PRs.
• Human reviewers approve the HAL contract, numerical tolerances, benchmark thresholds, supported-platform claims, and final OpenCL removal.
• Rayon retirement is outside this roadmap.

automation:
  eligible: false
  executor_profile: tembo
  readiness: rollup
  primary_repository: TinyChain-Inc/ha-ndarray
  base_branch: main
  writable_scope: []
  prerequisites: ["#34", "#41", "#42", "#43", "#44", "#45", "#46", "#47", "#48", "#49", "#39", "#50", "#22", "#40"]
  concurrency_group: cubecl-hal-program
  required_commands: []
  external_actions: none
  draft_pr_only: true
  completion_authority: human-program-review

Start with the existing ocl/OpenCL backend and host/Rayon execution as the independent correctness oracle, then review roadmap prerequisites #34 and #41–#50. Done requires CubeCL parity, cached compilation, supported accelerator and browser/WASM validation, and a human-approved OpenCL retirement decision while retaining the Rayon fallback.