Expose the LLM request router to split-cluster workers
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Describe the bug
The self-managed multi-cluster topology currently advertises llm-request-router.nvcf.svc.cluster.local:50071 to workers. That address is local to the control-plane cluster and is not resolvable or reachable from a separate compute cluster.
The local multi-cluster tooling has no LLM request-router alias Service or Endpoints in the compute cluster, no port-50071 k3d mapping, and no Gateway listener or TCPRoute for the router. The current BDD does not launch an LLM worker, so it does not exercise this connection.
Expected behavior
Split-cluster workers receive an LLM request-router address that resolves and is reachable from the compute cluster, and local multi-cluster tests verify the connection path.
Suggested scope
- Expose TCP port 50071 from the control-plane cluster through a Gateway and TCPRoute, load balancer, or equivalent supported path.
- Add the required k3d host-port plumbing for local multi-cluster testing.
- Add a compute-cluster alias Service and Endpoints, or define and document an external DNS endpoint.
- Keep tests/bdd/fixtures/self-managed-local-bdd-multi.yaml and the raw seed in tests/bdd/godog_test.go aligned.
- Add an end-to-end test that launches an LLM worker across clusters.
- Document the split-cluster network, DNS, routing, and TLS requirements.
Context
Follow-up to #678. That pull request wires the configured address through the NVCF API, but intentionally does not add cross-cluster network exposure.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/bdd/fixtures/self-managed-local-bdd-multi.yaml and the raw seed in tests/bdd/godog_test.go, then trace the existing split-cluster routing setup described in the issue. Verify the selected exposure path, k3d port mapping, compute-cluster aliasing, and required DNS, routing, and TLS behavior. Done means an LLM worker connects across clusters and the end-to-end BDD test passes with the network requirements documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100