[Tracking] Constrained-environment memory and layer-offload roadmap

@chilo-ms is already working on this.

Since Aug 26, 2026.

## Background
This continues the roadmap documented alongside the annotation-based partitioning
work (#27595, #28028, #28623, #28903). See docs/annotated_partitioning/
future_directions_constrained_env.md for the full design notes.

Goal: make ORT as easy to use as ollama/llama.cpp for running large models on
memory-constrained devices — automatic layer offloading without requiring exact
node annotations, and eventually static/pre-planned memory usage close to
llama.cpp's model.

## Status

**Near-term, low effort, high value:**
1. Name-based substring matching (`session.name_based_layer_assignment`,
`SubstringMatcher`) — **DONE**, shipped in #28903.
2. **Precise per-node memory/workspace estimation** — **NOT STARTED**, next
actionable step.
- Add `OrtKernelWorkspaceEstimateFunc` (Level 1 — at partitioning time,
before a kernel instance exists; used both in-tree and via the plugin
host bridge, i.e. the layer that lets out-of-tree/plugin EPs call into
the same estimation logic) and `DeclareWorkspaceRequirements()`
(Level 2 — after kernel creation).
- Resolve the chicken-and-egg problem: workspace size is only known
after a kernel exists, but EP assignment happens before kernel
creation (see doc §"The Chicken-and-Egg Problem" for resolution
options).
- Bypass the 1.5x safety multiplier in `IResourceAccountant` (ORT's
per-node memory-budget tracker used during partitioning) per-kernel
once an estimate is available; retain it as fallback for
un-instrumented kernels and dynamic/unknown shapes.
- ⚠️ Touches the plugin EP C ABI (`OrtEp` / `PluginExecutionProvider`,
see `ep_plugin_provider_interfaces.cc`) — a sensitive, versioned
boundary; coordinate with plugin EP owners.

**Mid-term, medium effort — NOT STARTED:**
3. Auto-partitioning with memory budget only — requires new design work
(layer discovery/boundary detection is NOT solved by the existing
per-node `IResourceAccountant`; needs explicit decisions on ordering
and atomicity before implementation). Can proceed independently of #2.
4. Static allocation mode (doc's Phase A: workspace pre-declaration +
Phase B: arena bypass for static-shape models). Depends on #2's
Level-2 workspace declaration.
- ⚠️ Requires an upfront workspace-declaration and planning phase
during `FinalizeSessionState()` / allocation planning (`allocation_planner.cc`),
plus execution-plan/frame state and kernel workspace retrieval changes.
Existing activations already bypass the arena via `BFCArena::Reserve()`;
undeclared/dynamic workspace must retain the current arena fallback.

**Long-term, high effort, ollama-parity — NOT STARTED (new work, not
scoped by the original design doc's two Directions):**
5. Layer prefetch pipeline (stream weights CPU↔GPU during execution).
6. GenAI integration (KV-cache-aware planning + continuous batching).
- ⚠️ KV-cache/continuous-batching logic lives in the separate
microsoft/onnxruntime-genai repo — this item likely needs
coordinated PRs across both repos (author annotation, not from the
source doc).

## How to pick this up
Sub-issues will be linked here as each item above is picked up — comment
on this issue or self-assign to claim one.

## Reference
Full design notes: docs/annotated_partitioning/future_directions_constrained_env.md