rossoctl / rossoctl/serverless-harness

P5b: deployment model — ScaledJob → elastic pod pool

Open
#240 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
TypeScript
Stars
1
Forks
7
Avg merge
12h 59m
Merged PRs (30d)
71

Description

Split out of #220. This is the actual density win — deliberately excluded from the P5 session-isolation slice (P5 spec §6, bullet 1).

Blocked by #239. Multiplexing sessions into shared pods is only safe once a turn's identity comes from the request rather than from process-global state; today harness/src/run-turn.ts:310-312 seeds ANTHROPIC_API_KEY write-once-if-absent, so co-tenanted sessions would authenticate as whichever one arrived first. #239 removes that.

Scope

  • ScaledJob → elastic pod pool. Replace one-pod-per-run with a warm pool that multiplexes N sessions per process.
  • Measure, don't assume. #220 claimed ~100× density from reading the code. Produce real pod-count and activation-latency numbers against the current baseline before the design is settled.
  • Overload semantics move tier. #55 (closed — "Sync-path saturation: bounded wait + 503 Retry-After per spec 4.3") solved saturation at the pod level. With N sessions per pod, saturation becomes a session-level concern, so the bounded-wait/Retry-After behaviour needs re-deriving rather than inheriting.

Prior art already written

A single-tenant prototype of the density half exists and is worth starting from. It is parked in a local stash (220 superseded single-tenant concurrency prototype (pre-P5)), not pushed — ask before assuming it is available. Reusable pieces:

File What
packages/knative-server/src/capacity.ts capacity accounting
packages/knative-server/src/leaf-worker.ts long-lived worker loop
packages/knative-server/test/capacity.test.ts capacity tests
deploy/knative/leaf-worker-deployment.yaml pool as a Deployment
deploy/knative/leaf-worker-scaledobject.yaml KEDA ScaledObject (vs ScaledJob)
scripts/load-driver.sh, scripts/sustained-load.sh load generation

⚠️ The credential half of that prototype is superseded — do not revive it. Its seedCredentialEnvVars() changes to run-turn.ts / server.ts / leaf-job.ts, its docs/adrs/0032-multi-session-concurrency.md, and harness/test/concurrent-sessions.test.ts (cases "does not overwrite an existing ANTHROPIC_API_KEY" and "seeds ANTHROPIC_API_KEY from ANTHROPIC_AUTH_TOKEN") assert exactly the behaviour ADR-0032 reverses. Under mixed tenancy the seeded environment is the identity leak, not the fix for it. The density files above never depended on that premise.

Relationships

  • Split out of #220, alongside #239 and the injector dependency.
  • Blocked by #239.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

First resolve the #239 dependency, then read packages/knative-server/src/capacity.ts and leaf-worker.ts plus the related capacity tests. Run scripts/load-driver.sh and scripts/sustained-load.sh against the current baseline, using the deployment manifests as the scaling entry points. Done means measured pod-count and activation-latency improvements, with session-level overload behavior re-derived and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, shell, typescript
Domain
backend, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.