rossoctl / rossoctl/serverless-harness
P5b: deployment model — ScaledJob → elastic pod pool
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 7
- Avg merge
- 12h 59m
- Merged PRs (30d)
- 71
Description
Split out of #220. This is the actual density win — deliberately excluded from the P5 session-isolation slice (P5 spec §6, bullet 1).
Blocked by #239. Multiplexing sessions into shared pods is only safe once a turn's identity comes from the request rather than from process-global state; today harness/src/run-turn.ts:310-312 seeds ANTHROPIC_API_KEY write-once-if-absent, so co-tenanted sessions would authenticate as whichever one arrived first. #239 removes that.
Scope
ScaledJob→ elastic pod pool. Replace one-pod-per-run with a warm pool that multiplexes N sessions per process.- Measure, don't assume. #220 claimed ~100× density from reading the code. Produce real pod-count and activation-latency numbers against the current baseline before the design is settled.
- Overload semantics move tier. #55 (closed — "Sync-path saturation: bounded wait + 503 Retry-After per spec 4.3") solved saturation at the pod level. With N sessions per pod, saturation becomes a session-level concern, so the bounded-wait/
Retry-Afterbehaviour needs re-deriving rather than inheriting.
Prior art already written
A single-tenant prototype of the density half exists and is worth starting from. It is parked in a local stash (220 superseded single-tenant concurrency prototype (pre-P5)), not pushed — ask before assuming it is available. Reusable pieces:
| File | What |
|---|---|
packages/knative-server/src/capacity.ts |
capacity accounting |
packages/knative-server/src/leaf-worker.ts |
long-lived worker loop |
packages/knative-server/test/capacity.test.ts |
capacity tests |
deploy/knative/leaf-worker-deployment.yaml |
pool as a Deployment |
deploy/knative/leaf-worker-scaledobject.yaml |
KEDA ScaledObject (vs ScaledJob) |
scripts/load-driver.sh, scripts/sustained-load.sh |
load generation |
⚠️ The credential half of that prototype is superseded — do not revive it. Its
seedCredentialEnvVars()changes torun-turn.ts/server.ts/leaf-job.ts, itsdocs/adrs/0032-multi-session-concurrency.md, andharness/test/concurrent-sessions.test.ts(cases"does not overwrite an existing ANTHROPIC_API_KEY"and"seeds ANTHROPIC_API_KEY from ANTHROPIC_AUTH_TOKEN") assert exactly the behaviour ADR-0032 reverses. Under mixed tenancy the seeded environment is the identity leak, not the fix for it. The density files above never depended on that premise.
Relationships
- Split out of #220, alongside #239 and the injector dependency.
- Blocked by #239.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
First resolve the #239 dependency, then read packages/knative-server/src/capacity.ts and leaf-worker.ts plus the related capacity tests. Run scripts/load-driver.sh and scripts/sustained-load.sh against the current baseline, using the deployment manifests as the scaling entry points. Done means measured pod-count and activation-latency improvements, with session-level overload behavior re-derived and tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, shell, typescript
- Domain
- backend, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100