rossoctl / rossoctl/serverless-harness
E8: run VM density + saturation (both arms) and record the result
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 7
- Avg merge
- 12h 59m
- Merged PRs (30d)
- 71
Description
Summary
Run E8 — VM density and saturation (P6 §5.2) — and record the result. The spec amendment and driver
work are #254; this issue is the run itself, which #254 explicitly does not cover.
Splitting them is deliberate. #254 lands a two-arm methodology (capacity + realism) and the driver
changes that make a knee reachable; it can be reviewed and merged without any hardware booked. The run
consumes that methodology, produces the number, and is where a bound=not-observed outcome has to be
adjudicated rather than merged. Keeping them in one issue is what let the previous calibrated run be
recorded as a density figure when it was stub-sleep-bound.
Preconditions — all of them, checked before booking anything
| # | Gate | Why it blocks |
|---|---|---|
| 1 | #254 merged | Without the capacity arm the ladder tops out at the admission cap W×S=32, a config choice, and the run reproduces bound=not-observed |
| 2 | PR #253 merged | /turn's tool calls must reach the sandbox pool. A run where tools execute locally measures a harness with no hands tier — and the counting probe is what catches it |
| 3 | #239 merged if mixed-subject rungs are claimed | P6 §3.6/§7: single-subject rungs exercise every mechanism this slice adds and need no P5. Mixed subjects and the isolation claim need it. Decide which run this is before starting |
| 4 | Model stub deployed with a tool-call profile (§5.4) | A text-only stub means no session reaches the sandbox and the density number silently excludes the hands tier |
| 5 | Duty basis pinned to one row of §2.3 | Default E6/OCP: duty = 0.061–0.079, provision at 12.6–16.5. The driver asserts K ≥ ceil(W × S × duty) with duty and the ratio from the same row |
Gate 3 is a scoping decision, not a dependency to wait on. Per P6 §3.6, P5 slipping delays a claim,
not the work — so a single-subject E8 is runnable the moment gates 1, 2, 4, 5 hold, and should not be
held behind #239.
What to run
Both arms from #254's amended §5.2, each recorded separately and never merged into one figure:
- capacity arm — trivial exec, fast stub profile, sandbox tier non-binding and non-competing.
Sweep offered concurrency upward withSH_WORKERS × SH_TURNS_PER_WORKERand
KAGENTI_SANDBOX_CAPset well past the expected knee. Reports an upper bound on the harness
tier — explicitly not a deployable density. - realism arm — calibrated duty (
e6-ocpbasis), stub profile 300/12/64. This is the deployable
number, sandbox-shaped by construction.
W=1 is a rung, not an aside: it makes "one process would be simpler" a data point (§5.2).
Knee detection reuses E6's sustained-decline detectKnee with degradeX=2 against a warm C=1
baseline, and the knee is reported as a floor, not a ceiling (EXPERIMENTS.md:120).
Acceptance
- Both arms run; each arm's stub profile recorded and pinned via
assert_stub_pinned, and the
arm label verified against what/profilereturns - Per rung: throughput, p50/p95, event-loop lag p99, RSS, per-turn worker CPU, sandbox-container
CPU, lease saturation, over-admission events, spurious429s - Duty basis row and connections per session recorded on every rung, not only sticky ones
- The knee is attributed — worker CPU vs RSS vs replenishment vs admission cap — not just located
- If the capacity arm still returns
bound=not-observed, say so and name what bound first; do not
report a floor as a ceiling -
% of ideal throughputis not reported as a capacity figure when the ideal is stub-sleep-bound - Single-subject vs mixed-subject scope stated in the record, with the isolation claim made only
if #239 landed - Results in
deploy/vm/EXPERIMENTS.md, pointer added fromdocs/experiment-results.md(§5.6)
Open question to settle during the run
Is event-loop lag trustworthy as the capacity arm's headline diagnostic? It read ~11 ms flat at
every rung including c=1, where the tier is nearly idle — which looks like a sampling or timer
artefact. §5.2 makes it the metric that attributes a knee to worker CPU, so verify it against a
deliberately CPU-starved worker before relying on it. If it cannot be trusted, per-turn worker CPU
(#254 §2e) carries the attribution alone and the record should say so.
Related: #254 (spec + driver), PR #253 (routing prerequisite), #239 (mixed-subject scope), E9 run.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
First verify gates #254, PR #253, the stub profile, and the pinned §2.3 duty row before booking the run. Use the amended §5.2 procedure for separate capacity and realism arms, then record per-rung metrics and attribution in deploy/vm/EXPERIMENTS.md and add the pointer to docs/experiment-results.md.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- infrastructure, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100