es-ude / es-ude/OnDeviceTraining
arithmetic: flex-point / block-floating-point tensor format for FPU-less MCUs
- Dominant language
- C
- Stars
- 1
- Forks
- 3
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 8
Description
## Motivation
Speed up training/inference on FPU-less MCUs (Cortex-M0/M3, RV32 without the F extension). Today every SYM requantization multiplies by an arbitrary **float** scale (`float_value ≈ int32_value * scale`) — under soft-float emulation that is expensive precisely on the targets that need integer math most. The kernels can be integer-only in accumulation and still pay float costs at every scale boundary.
## Idea
Flex-point / block-floating-point family: per-tensor shared exponent with a **power-of-two scale**, so every rescale becomes an integer shift and elementwise ops/accumulation stay pure-integer end-to-end. Natural fit with the declared-arithmetic split (`arithmetic_t`) and the integer-only-training goal (#308/#309/#310).
## Required first step: literature review (repo rule — no homegrown numerics on simulation alone)
- Flexpoint (Köster et al., NeurIPS 2017) — shared-exponent training, exponent management (AutoFlex)
- Block-floating-point training literature (MSFP and successors)
- Deutel et al., IEEE TCAD 44(4) 2025 (arXiv:2407.10734) — the repo's standing FQT reference; how power-of-two scales relate to its integer-only accumulation scheme
- Survey what dynamic-exponent management costs during *training* (grad magnitudes drift; a stale shared exponent silently clips or starves)
## Key design decision
New dtype vs. a **power-of-two-scale constraint on the existing SYM types**. The latter reuses every SYM kernel unchanged and turns scale multiplies into shifts — likely the cheap path; the existing "wide SYM requires scale=1" rule is the degenerate case of exactly this idea. A new dtype only pays off if per-block (not per-tensor) exponents are wanted.
## Deliverable
Design memo with lit-review + cost model (shift vs. soft-float multiply on a representative FPU-less target), then an implementation slicing proposal. Measurement per repo methodology: ≥10-seed sweeps once anything trains.
Contributor guide
Research direction
No files or tests are named. Start with the cited Flexpoint, block-floating-point, and Deutel et al. literature, then inspect the existing SYM types and declared-arithmetic design referenced in the issue. Done means a design memo covering the literature review, exponent-management risks, a shift-versus-soft-float cost model, and an implementation slicing proposal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- embedded-iot, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100