feat(guardrails): multi-lane inference pool for the local-model guardrail

Context

PR #999 ships the local-model guardrail MVP with ONE inference lane: a single ONNX session behind Semaphore(1) + Mutex, measured at ~50 inferences/s (p50 ≈ 19 ms per ~35-token window on a 12-core avx2+vnni host). The acceptance shape spends two inferences per request (input + output pass), so the single lane saturates around ~25 guardrail-active requests/s and all cores' requests queue on it.

Design (decided in the MVP review; recording it here so it doesn't rot in the PR thread)

Scale throughput by lanes, not by intra-op threads:

• GUARDRAIL_LOCAL_MODEL_LANES=N (env, default 1 — same experimental env surface as the existing knobs).
• N sessions, ONE process-wide admission semaphore with N permits, and a centralized free-list of sessions: a granted task takes whichever session is idle and returns it afterwards.
• Explicitly NO worker↔session binding. Sessions are stateless loaded-model instances and fully interchangeable, so binding buys no correctness or locality; TPC workers receive connections unevenly (kernel accept distribution), so a static per-core split would idle some lanes while others queue. The centralized queue load-balances by construction. The only 1:1 binding that may exist is in the hard-isolation variant below, and it is inference-thread↔session — never request-worker↔session.
• Keep intra_op = 1 per session: short windows (~35 tokens) parallelize poorly across intra-op threads; the throughput axis is lane count, and one thread per lane is the best per-core efficiency.
• Optional hard-isolation variant (only if soft partitioning proves insufficient): replace spawn_blocking with a dedicated pinned inference thread pool, each thread OWNING one session (drops the mutex), consuming one bounded task channel; pin business workers and inference threads to disjoint core sets.

Cost / constraint to record

Each lane pays its own ~190 MiB int8 weight copy: ort 2.0.0-rc.13 types Session::run(&mut self), which forbids the shared-weights concurrent-Run form the ONNX Runtime C API documents as thread-safe, and aisix-guardrails is #![forbid(unsafe_code)]. Paths back to a single weight copy, any one of which closes this: an upstream &self run signature, a thin unsafe shim crate outside the forbid boundary, or the sidecar deployment form. Until then, N lanes ≈ N × weight memory (4 lanes ≈ +600–700 MiB) — size against the target host.

Acceptance

• Default (LANES unset / 1) behaves exactly as today.
• Throughput scales ~linearly to N on the acceptance workload (measure and record, as PR #999 did for the single lane).
• Memory increment per lane measured and documented.
• Cancellation semantics preserved: permits must keep travelling into the blocking closure (PR #999 audit finding 2).

References

• PR #999 (MVP + audit thread; module-doc Scaling notes in crates/aisix-guardrails/src/local_model.rs)
• api7/AISIX-Cloud#1331 (second-tier guardrail umbrella; this issue is one dimension of its productization pass)

Start with crates/aisix-guardrails/src/local_model.rs and the MVP implementation and audit notes in PR #999. Trace the current single-lane session, semaphore, mutex, and spawn_blocking flow, then run the acceptance workload used by PR #999. Done means the default remains unchanged, throughput and per-lane memory are measured and documented, and cancellation semantics remain preserved.