es-ude / es-ude/OnDeviceTraining
layer: DenseNet-style dense blocks for 1D and 2D CNNs
- Dominant language
- C
- Stars
- 1
- Forks
- 3
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 8
Description
## Feature
DenseNet-style dense blocks for 1D and 2D CNNs: layer *i* consumes the channel-concat of all previous feature maps within the block (growth rate k); transition layers (kernel-size-1 conv + pooling) between blocks.
## Blockers
- **#329 (topology epic)** — multi-input consumption and the channel-dim **Concat** op (forward gather, backward dx slicing) are exactly epic deliverables 4–5.
- 2D variant additionally blocked by **#330**.
## Design notes
- **Memory is the crux**: dense connectivity is the most buffer-hungry topology, and a naive per-concat copy multiplies activation memory. Zero-copy concat is out of reach of the current tensor model (single data pointer; the `orderOfDimensions` machinery cannot express a gather from disjoint buffers) — it would require the buffer planner to lay out a block's activations contiguously. Decide **copy-first vs. planner-first** explicitly; on MCU this is the difference between feasible and not.
- SYM configs: concat of differently-scaled wires either requantizes to a common scale (accuracy cost) or needs per-slice scale tracking (outside the current per-tensor quantization model) — same design round as #329's quantized-merge semantics.
## Acceptance
A 1D dense-block example with PyTorch twin, ≥10 seeds; memory breakdown reported via the existing `ODT_MEM_PROFILE` facility (the interesting number *is* the activation footprint).
Contributor guide
Research direction
Start by reading blockers #329 and #330, then review the existing ODT_MEM_PROFILE facility and the current tensor and convolution abstractions. The work is done when a 1D dense-block example matches a PyTorch twin across at least 10 seeds, includes the required transition behavior, and reports the activation-footprint breakdown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, pytorch
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100