es-ude / es-ude/OnDeviceTraining

Epic: classic Block Floating Point (BFP) — new arithmetic type + dtype, every layer, sweepable

Open
#410 0 comments 0 reactions 0 assignees View on GitHub
enhancement research
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

Anchor issue for the BFP epic (design ratified 2026-07-29; local spec: `docs/superpowers/specs/2026-07-29-block-floating-point-design.md`, D1–D12).

## Goal

Classic block floating point (blocks of two's-complement integer mantissas sharing one power-of-two exponent) as a first-class **storage dtype** (`qtype_t BFP`) **and compute arithmetic** (`ARITH_BFP`), supported by every layer, to run accuracy-over-training sweeps against the FLOAT32 and sym baselines. Published classic-BFP *training* on Cortex-M-class hardware does not exist — this epic is the framework side of that research niche.

## Ratified decisions (condensed)

- **End-to-end training**: forward + backward + gradients; sweep = accuracy/loss curves over epochs.
- **Fully parametric sweep axes**: weight-block size, wire-block size (independent, incl. per-tensor extreme), `mantissaBits`, `exponentBits`, rounding (HALF_AWAY / SR via existing knob).
- **Fully native incl. Softmax + Norm backwards** (research package: I-BERT anchor for integer softmax; own error analyses + documented deviations). Exceptions by decision: Dropout/Flatten non-native, losses fake-quant (native CE as optional stretch).
- **Wires fully blocked**; compute blocking is op-local (exponents recomputed before every dot product — resolves the zero-copy-transpose hazard).
- **Exponent saturation, not abort** (value domain; the `exponentBits` axis measures saturation). Code-domain packing keeps the abort discipline.
- **FLOAT32 raw intermediate**: int32 mantissa MACs within a block, exact `2^E` fold, float32 cross-block accumulation — no int64 anywhere.
- **ConvT1d gather-formulated** under `ARITH_BFP` so the int32-block-partial contract holds uniformly across the GEMM family.
- Rides the group-quant substrate (storage-order groups, `{1,0}` per-tensor sentinel); metadata = 1 B/group u8 exponent vs 4 B float scale — the accuracy-per-byte headline.

## PR slicing

| PR | Content | Status |
|---|---|---|
| 1 | Dtype core: `BFP` qtype, complete 7×7 conversion matrix, owner chain, unified ODTS v5, float-bridge derivation, fake-quant e2e capstone | ✅ merged (#408) |
| 2 | `ARITH_BFP` funnel + GEMM-family forward (both operands blocked, headroom guard `g ≤ 2^(33−2m)`) + wire allocators | ✅ merged (#413) |
| 3 | GEMM-family backward + grad-storage knob + optimizer enablement | ✅ merged (#422; hardening stacks #428/#430) |
| 4 | Pools + Relu/Dropout/Flatten block semantics + loss fake-quant arms | ✅ merged (develop fast-forward 2026-09-07) |
| 5 | LayerNorm/GroupNorm native fwd+bwd (research package I) | ✅ merged (#435) |
| 6 | Softmax native, I-BERT adaptation + CE coupling (research package II) | ✅ merged (#439, develop fast-forward 2026-09-09) |
| 7 | Sweep integration: HAR trainer, run_matrix configs, log schema, memory accounting | open |

Ordering: 1→2→3 strict; 4/5/6 after 2, mutually independent; 7 stages after 3 and extends after 5/6.

## Carry-overs into PR2 (from PR1 reviews) — all six shipped in PR #413

- n>256 grouped pack fixture (multi-chunk path empirically unpinned)
- `requantBfpTensor` aliasing fail-fast once call sites multiply
- Wire-allocator fail-fasts get a guided "BFP wires arrive with epic PR2" message (replacing generic `Unknown QType!`)
- Perf pass on the FLOAT32↔BFP cells (bounce buffer, per-element `bfpGroupScale` recompute)
- Capstone `rngSetSeed` hygiene (+ RNG link in userAPI tests)
- FEATURES.md: grouped-ASYM-target deny applies to BFP sources too (clarifying clause)

## Validation mechanism (established in PR1)

Fake-quant twins via the float bridge = simulated-quantization reference curves; every native kernel validates against its twin; power-of-two-scale configs must run **bit-identical** to the grouped-SYM path (the PR2/PR3 group-quant twin mechanism as BFP acceptance test).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Read docs/superpowers/specs/2026-07-29-block-floating-point-design.md and the PR-slicing table first. For the remaining work, inspect the HAR trainer and run_matrix configuration entry points, then define the log schema and memory accounting needed for the sweep. Done means PR7 supports the configured BFP accuracy/loss sweeps and records their results.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
embedded-iot, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.