Performance improvements: HTTP/2 flow control, embedding reload churn, fingerprint log gating

Performance improvements: HTTP/2 flow control, embedding reload churn, fingerprint log gating

Full implementation: https://github.com/jaivial/jcode-perf (mirror of the fork https://github.com/jaivial/jcode), built on top of v0.67.0. The changes are small, independent, and covered by unit tests.

1. HTTP/2 flow-control windows + warm connection pool

crates/jcode-provider-core/src/lib.rs (shared_http_client, fresh_transport_client)

• Raised http2_initial_stream_window_size from the 64 KiB spec default to 1 MiB and http2_initial_connection_window_size to 4 MiB. A long conversation turn is 1-2 MiB of JSON; with the default window the client stalled for a WINDOW_UPDATE round trip on every 64 KiB boundary, which is hundreds of milliseconds of pure stall per turn over a high-latency link.
• Raised pool_idle_timeout from 90 s to 300 s. Idle gaps between turns routinely exceed 90 s, so the next turn used to pay a full TCP+TLS handshake plus proxy/upstream connection setup on the critical path. HTTP/2 keep-alive pings keep the pooled connection healthy while idle.
• fresh_transport_client() matches the new flow-control windows so retried requests upload just as efficiently.

2. Adaptive embedding-model idle unload (stops reload churn)

crates/jcode-base/src/embedding.rs, crates/jcode-app-core/src/server.rs

On a real deployment the all-MiniLM-L6-v2 model was loaded 10x / unloaded 9x in ~65 minutes: the 60 s idle-unload default was shorter than the gap between memory operations, so every few minutes the server paid a ~500 ms model load plus a fresh ~90 MB mapping for an embed the just-unloaded model would have served.

Now every unload records last_unload_at; if a reload follows within 10 minutes the idle threshold doubles (bounded at 16x and 30 minutes), so the model stays resident for the working session that keeps using it. A reload long after an unload resets the backoff, so a genuinely idle server still returns the memory. The server's idle monitor recomputes the effective threshold each tick.

3. Canonical-input fingerprint logging gated behind JCODE_TRACE

crates/jcode-provider-core/src/fingerprint.rs + all ten provider runtimes

PROVIDER_CANONICAL_INPUT serialized the entire prompt on every request: clone the multi-MB transcript into a serde_json::Value, serialize it several more times, and SHA-256 every message. Measured ~18 ms of CPU plus multiple multi-MB transient allocations per 3.9 MiB turn, on the request hot path in front of the HTTP send.

log_provider_canonical_input now returns before any serialization unless JCODE_TRACE is set, and every provider's snapshot build is skipped entirely. Normal operation pays nothing. Re-enable with JCODE_TRACE=1 jcode.

Verification

• 132 unit tests pass (provider-core 117, jcode-base embedding 10, fingerprint gate 5); workspace compiles clean.
• Release binary validated end-to-end against a mock OpenAI-compatible provider under isolated JCODE_HOME: full conversation round-trip, correct auth/body, connection reuse confirmed (HTTP connection established in 1ms), and the JCODE_TRACE gate verified in both directions (diagnostic present with the flag, absent without).
• Measured latency split (jcode -> local gateway -> upstream): 1-token ~1.25 s TTFB, 10K-token 1.6 s, 50K-token 3.2 s, back-to-back consistent (no connection churn). The logged 1.2-5.5 s "HTTP connection established" is dominated by the fixed gateway/upstream floor; KV-cache telemetry shows 95-100% cache reads on long sessions.

Full details, including the measured evidence and build/install notes: docs/PERF_OPTIMIZATIONS_FORK.md in the linked repo.

Commits: a9c9754 (transport), f4a4b26 (embedding), 8fe5217 (fingerprint gating), plus two doc commits (b75c937, c05ae63).

Review the referenced commits and compare their changes in crates/jcode-provider-core/src/lib.rs, crates/jcode-base/src/embedding.rs, crates/jcode-app-core/src/server.rs, and crates/jcode-provider-core/src/fingerprint.rs plus the provider runtimes. Run the stated provider-core, embedding, and fingerprint unit tests first; completion means the changes are integrated, the workspace compiles, and the documented end-to-end and JCODE_TRACE checks pass.