es-ude / es-ude/OnDeviceTraining

Implement TrainingImplementationProviders (Float32, SymInt32, Asym)

Open
#60 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

## Context

Sub-issue of #58.

**Blocked by:** #59 (scaffolding)
**Blocks:** #62 (ir2c), #64 (E2E tests)

## Goal

Implement `TrainingImplementationProvider` protocol for all three quantization types.

## Protocol (defined by creator)

```python
class TrainingImplementationProvider(Protocol):
def model_attributes(self, sub_graph: DataGraph) -> AttributeMapping: ...
def training_function(self) -> DataGraph: ...
def optimizer(self) -> DataGraph: ...
def loss(self) -> DataGraph: ...
```

## Provider Architecture

### Float32 and SymInt32
Native layer ops — `model_attributes()` maps directly to C function symbols:
- `forward_fn = "linearForwardFloat"` / `"linearForwardSymInt32"`
- `backward_fn = "backwardFloat"` / `"backwardSymInt32"`

### Asym
No embedded hardware supports ASYM natively. The `AsymProvider` inserts **conversion nodes** at layer boundaries:
```
ASYM tensor → convertTensor(ASYM→Float32) → linearForwardFloat → convertTensor(Float32→ASYM) → ASYM tensor
```
The IR graph produced by the Asym provider contains explicit conversion operations. `ir2c` sees only nodes it generates 1:1 — it doesn't need to know about the ASYM strategy.

## Per-Provider Scope

Each provider must implement:
- `model_attributes()` — C symbols + conversion nodes (Asym) per layer type
- `training_function()` — dataloader config, epoch/batch loop, LR scheduling, memory defragmentation
- `optimizer()` — SGD: learning rate, momentum, weight decay
- `loss()` — CrossEntropy (with softmax coupling), MSE

### Layer Support Matrix

| Layer | Float32 | SymInt32 | Asym (via conversion) |
|-------|---------|----------|----------------------|
| Linear forward/backward | ✅ | ✅ | ✅ |
| ReLU forward/backward | ✅ | ✅ | ✅ |
| Softmax forward/backward | ✅ | ✅ | ✅ |
| CrossEntropy loss | ✅ | ✅ | ✅ |
| MSE loss | ✅ | ✅ | ✅ |
| SGD optimizer | ✅ | ✅ | ✅ |

## Acceptance Criteria

- [ ] `Float32TrainingProvider` implemented and tested
- [ ] `SymInt32TrainingProvider` implemented and tested
- [ ] `AsymTrainingProvider` implemented and tested — inserts conversion nodes correctly
- [ ] All providers return valid sub-graphs accepted by `apply_training_provider`
- [ ] C symbol names match actual function names in the C framework
- [ ] Conversion node insertion tested for Asym: correct placement, correct source/target types

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.