feat(llm-routing): parse routing expressions and apply tuning overlays in the router
@along-2017 is already working on this.
Since Sep 3, 2026.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Description
Part of #536 (per-model routing configuration).
Teach the request router to accept a routing method expression, an algorithm
name plus scalar tuning parameters, in the routing-method header. Parse it as a
profile of RFC 8941 Item with Parameters. Resolve the algorithm through the
existing configured/default/request-algorithm chain, clone the resolved base
configuration, and overlay the supplied parameters; omitted fields keep their
static values. Reject a semantically invalid expression (unknown method or
parameter, incompatible parameter, invalid value, inert combination) with HTTP
400 before provider selection, using the existing rejection logging shape.
Method-only values keep today's behavior. This ticket builds definitions per
request without identity reuse; production enablement waits for the cache
ticket.
Definition of Done
- Well-formed expressions for every supported algorithm parse and route
with the requested overlays applied; omitted fields keep static values - Method-only values behave exactly as today
- Semantically invalid expressions return 400 before provider selection
with a logged reason - Every direct scalar load-balancer configuration field is supported and
its algorithm applicability enumerated - Table-driven tests cover grammar acceptance, rejection, and overlay
semantics
Resources
- Parent: #536
- Blocked by: none
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.