es-ude / es-ude/OnDeviceTraining

layer: DenseNet-style dense blocks for 1D and 2D CNNs

Open
#332 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
1
Forks
3
Avg merge
1d 1h
Merged PRs (30d)
8

Description

## Feature

DenseNet-style dense blocks for 1D and 2D CNNs: layer *i* consumes the channel-concat of all previous feature maps within the block (growth rate k); transition layers (kernel-size-1 conv + pooling) between blocks.

## Blockers

- **#329 (topology epic)** — multi-input consumption and the channel-dim **Concat** op (forward gather, backward dx slicing) are exactly epic deliverables 4–5.
- 2D variant additionally blocked by **#330**.

## Design notes

- **Memory is the crux**: dense connectivity is the most buffer-hungry topology, and a naive per-concat copy multiplies activation memory. Zero-copy concat is out of reach of the current tensor model (single data pointer; the `orderOfDimensions` machinery cannot express a gather from disjoint buffers) — it would require the buffer planner to lay out a block's activations contiguously. Decide **copy-first vs. planner-first** explicitly; on MCU this is the difference between feasible and not.
- SYM configs: concat of differently-scaled wires either requantizes to a common scale (accuracy cost) or needs per-slice scale tracking (outside the current per-tensor quantization model) — same design round as #329's quantized-merge semantics.

## Acceptance

A 1D dense-block example with PyTorch twin, ≥10 seeds; memory breakdown reported via the existing `ODT_MEM_PROFILE` facility (the interesting number *is* the activation footprint).

Contributor guide

Open the contributing guide

Research direction

Start by reading blockers #329 and #330, then review the existing ODT_MEM_PROFILE facility and the current tensor and convolution abstractions. The work is done when a 1D dense-block example matches a PyTorch twin across at least 10 seeds, includes the required transition behavior, and reports the activation-footprint breakdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, pytorch
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.