es-ude / es-ude/OnDeviceTraining

arithmetic: flex-point / block-floating-point tensor format for FPU-less MCUs

Open
#335 0 comments 0 reactions 0 assignees View on GitHub
enhancement research
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

## Motivation

Speed up training/inference on FPU-less MCUs (Cortex-M0/M3, RV32 without the F extension). Today every SYM requantization multiplies by an arbitrary **float** scale (`float_value ≈ int32_value * scale`) — under soft-float emulation that is expensive precisely on the targets that need integer math most. The kernels can be integer-only in accumulation and still pay float costs at every scale boundary.

## Idea

Flex-point / block-floating-point family: per-tensor shared exponent with a **power-of-two scale**, so every rescale becomes an integer shift and elementwise ops/accumulation stay pure-integer end-to-end. Natural fit with the declared-arithmetic split (`arithmetic_t`) and the integer-only-training goal (#308/#309/#310).

## Required first step: literature review (repo rule — no homegrown numerics on simulation alone)

- Flexpoint (Köster et al., NeurIPS 2017) — shared-exponent training, exponent management (AutoFlex)
- Block-floating-point training literature (MSFP and successors)
- Deutel et al., IEEE TCAD 44(4) 2025 (arXiv:2407.10734) — the repo's standing FQT reference; how power-of-two scales relate to its integer-only accumulation scheme
- Survey what dynamic-exponent management costs during *training* (grad magnitudes drift; a stale shared exponent silently clips or starves)

## Key design decision

New dtype vs. a **power-of-two-scale constraint on the existing SYM types**. The latter reuses every SYM kernel unchanged and turns scale multiplies into shifts — likely the cheap path; the existing "wide SYM requires scale=1" rule is the degenerate case of exactly this idea. A new dtype only pays off if per-block (not per-tensor) exponents are wanted.

## Deliverable

Design memo with lit-review + cost model (shift vs. soft-float multiply on a representative FPU-less target), then an implementation slicing proposal. Measurement per repo methodology: ≥10-seed sweeps once anything trains.

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start with the cited Flexpoint, block-floating-point, and Deutel et al. literature, then inspect the existing SYM types and declared-arithmetic design referenced in the issue. Done means a design memo covering the literature review, exponent-management risks, a shift-versus-soft-float cost model, and an implementation slicing proposal.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
embedded-iot, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.