aws / aws/jsii

Protocol pipelining via client-allocated object refs (POC: ~17-20% faster synth)

Open
#5,167 4 comments 0 reactions 0 assignees View on GitHub
feature-request p2
Dominant language
TypeScript
Stars
2.9k
Forks
267
Avg merge
1d 25m
Merged PRs (30d)
14

Description

## Context

Large CDK (Python) synths spend most of their time in the synchronous host↔kernel protocol: a strict request/response ping-pong where the host blocks on every `create`/`invoke`/`get`. Profiling (see #5166) showed each side idle ~half the time waiting for the other, while JSON serialization is only ~2% — so the lever is **overlapping round-trips**, not the wire format.

This issue tracks a working proof-of-concept of **protocol pipelining via client-allocated object references**, and the work needed to productionize it. Draft PR: (link below).

> Prototyped by **Kiro** (an AI agent), working with @mrgrain.

## Mechanism

- Any call whose declared return is a **reference type** (class/interface) is fired **without waiting**: the host mints the object id, sends it with the request, and uses a synthetic handle immediately.
- The kernel honors a client-supplied `objid` on `create`, and on `invoke`/`sinvoke`/`get`/`sget` **aliases** that id to whatever object the call actually produced — fresh or pre-existing. Multiple ids → one object is fine; kernel-side identity stays correct.
- The host drains acks lazily and only blocks at **true sync points**: value returns (e.g. reading a token string), callbacks, end of synth. A pending cap bounds outstanding requests to avoid pipe-buffer deadlock.
- ref-vs-value is decided from the generated binding's declared return type.

## Results (POC)

~2,200-resource synth, best-of-5, same runtime, **byte-identical CloudFormation output**:

| variant | wall |
|---|---|
| baseline | ~5.8–6.1 s |
| pipelined (create+invoke+sinvoke+sget) | ~4.7–4.9 s |

**~17–20% faster.** Findings:
- **create-only pipelining ≈ 0%** — interleaved synchronous invokes are barriers that drain the pipeline. Pipelining invokes too is what unlocks the win.
- **get/sget added little here** — instance `get`s in this app are all *value* reads (`bucket_name`, `queue_url`, `table_name` → token strings), which are irreducible sync points; only static `sget`s returned references.
- The remaining gap to the theoretical ~2× ceiling is the **kernel's serial work** (construct JS + dispatch), not the protocol. Pipelining overlaps the host with the kernel; once overlapped, you're kernel-work-bound.

## Remaining TODOs / gaps

Productionizing this is a real cross-cutting project:

- [ ] **Codegen instead of runtime introspection.** The POC reads the caller's return annotation at runtime (frame walk + `get_type_hints`, cached). Production should have **pacmak emit the return fqn** into the binding call (`jsii.invoke(self, "m", args, return_fqn=...)`) — faster, deterministic, and removes the frame-walk hack. Needed per target language.
- [ ] **Deferred error semantics.** A failed pipelined call surfaces at the next sync point, not the call site. Requests need tagging so errors map back to the originating call with a usable stack. The POC ignores errors on the drained path.
- [ ] **Callbacks.** Overrides (host ← kernel callbacks) are hard sync points; the pipeline must drain/coordinate around them. The POC asserts/raises if a callback appears mid-pipeline (the test app has none).
- [ ] **Object identity.** Client-minted ids mean two reads of the same underlying object yield two distinct host proxies (`a.prop is a.prop` can become False). Kernel-side identity is preserved; host-side needs a story (e.g. cache ref→proxy, or reconcile at sync points). Did not affect output here, but identity-sensitive code could differ.
- [ ] **Id-space / collision discipline.** The kernel must accept and trust client ids in a defined, non-colliding range (POC uses `fqn@<1e9+seq>`); formalize and validate.
- [ ] **Capability negotiation.** Gate on a handshake flag so old/new hosts and kernels interoperate; never send `objid` to a kernel that doesn't understand it.
- [ ] **Pending-cap tuning / backpressure.** POC uses a fixed cap of 64; needs principled backpressure to avoid pipe-buffer deadlock across platforms.
- [ ] **All language runtimes.** POC is Python-only; the kernel side is language-agnostic, but Java/.NET/Go hosts need the same client work.
- [ ] **Async (`begin`/`end`) and `del`** interactions with the pipeline are unexplored (absent in the test app).
- [ ] **Tests** for the kernel alias behavior and the host pipeline/drain/correctness; broader app coverage beyond the synthetic stack.

## Related
- #5166 — wire-protocol profiling and analysis (why JSON isn't the bottleneck; the ~2× ceiling).

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing Python POC and the pacmak binding-generation path around the jsii.invoke call, then inspect the kernel create/invoke/get alias handling. Run the existing synth benchmark and identify tests covering pipeline draining and correctness. Done requires agreed error, callback, identity, id-space, negotiation, backpressure, async, and multi-language behavior with kernel and host tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, go, java, python, typescript
Domain
backend-api-design, distributed-systems, performance, testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.