es-ude / es-ude/OnDeviceTraining

userApi: wire allocators hard-exit on packed SYM/ASYM — packed activation/dx wires structurally impossible

Open
#317 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

## Problem

All four wire allocators clone the wire's quantization through a hand-rolled two-arm switch that accepts only FLOAT32 and SYM_INT32 and `exit(1)`s ("Unknown QType!") on everything else:

- `initLayerOutputs` — `src/userApi/training_loop/calculate_grads/CalculateGradsSequential.c:205-219` (training-forward activation wires)
- `initGradTensor` — `src/userApi/training_loop/calculate_grads/CalculateGradsSequential.c:266-279` (backward dx wires + loss-grad seed)
- `initBufferOutput` — `src/userApi/InferenceApi.c:55-69` (inference activation wires)
- `initBufferInput` — `src/userApi/InferenceApi.c:93-110` (inference input staging buffer)

Concrete failure: declare a packed wire on any layer — e.g. `outputQ`/`propLossQ` = `quantizationInitSym(8, ...)`; nothing at config-build time rejects this (`layerQuantInitUniform`/`deepCopyQuantization` accept SYM/ASYM, `src/userApi/LayerQuant.c:10-73`) — then call `inference()` or `calculateGradsSequential()`. The process aborts during wire allocation, before any layer kernel runs (`initLayerOutputs` executes ahead of the forward loop, CalculateGradsSequential.c:46; `initBufferOutput` ahead of each `forward`, InferenceApi.c:134). Sub-byte packed activation and dx wires are therefore structurally impossible via every public training and inference entry point, independent of any layer's SYM/ASYM support.

The restriction is allocator-local, not systemic — the surrounding machinery is already packed-capable:

- payload sizing at all four sites routes through `calcNumberOfBytesForData` (the packed-aware ceiling authority; CalculateGradsSequential.c:201,262, InferenceApi.c:51,89)
- the general Q-cloner `getQLike` (`src/userApi/tensor/TensorApi.c:196-237`) clones SYM/ASYM (only BOOL fails fast by design)
- the executeOp funnel converts any non-BOOL operand dtype in its prologue and writes any non-BOOL target dtype in OUT_WRITE (`src/arithmetic/ExecuteOp.c`); the conversionMatrix is fully populated for SYM/ASYM (`src/tensor/TensorConversion.c:541-577`)

Epic #300's axis 6 (wire storage) lists the downstream packed-wire blockers — non-funnel MaxPool/AvgPool backward, ReLU backward on SYM codes, the FLOAT32-only loss head — but not this gate, which fires first and blocks the axis before any of those matter. Filing per the epic's boundaries-get-their-own-issue rule.

## Proposal

Replace the four hand-rolled q-clone switches with the existing `getQLike` (TensorApi.c:196) — it preserves width + rounding and resets the scale (fresh ungridded clone; the producing OUT_WRITE epilogue derives the grid, and for the input staging buffer `copyTensor`→`copyQuantization` restores the source qConfig including scale). Both files already include `TensorApi.h`.

Necessary but not sufficient: this removes the structural gate so packed wires can be worked layer by layer; end-to-end packed wires still need the axis-6 blockers (pool/ReLU backward funnel-ization, loss head) resolved. The axis-6 scoping memo should add this gate to its blocker list.

Refs: #300 (axis 6), #171/#172 (closed — packed clone/sizing support at the tensor layer)

Contributor guide

Open the contributing guide

Research direction

Start with the four allocator sites in src/userApi/training_loop/calculate_grads/CalculateGradsSequential.c and src/userApi/InferenceApi.c, then read getQLike in src/userApi/tensor/TensorApi.c and the packed sizing path. Exercise inference() and calculateGradsSequential() with packed wires; done means allocation no longer aborts on supported SYM/ASYM quantization, while the separate downstream blockers remain out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
embedded-iot
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.