Layr-Labs / Layr-Labs/d-inference
E2E flake on master: provider WebSocket disconnect under concurrent load → all requests 502
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 527
- Forks
- 114
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 111
Description
## Symptom
`TestIntegration_ConcurrentRequests` and `TestProfile_SingleProviderNonStreaming` fail intermittently on master with:
```
integration_test.go:554: "0" is not greater than "0" — at least some concurrent requests should succeed
status=502 {"code":"provider_error","message":"inference failed after 2 attempt(s): provider disconnected"}
```
The testbed provider drops its WebSocket under concurrent load; retries land on the same (sole) disconnected provider, so every request 502s.
## Evidence
- 4 of the last 5 master runs of `integration.yml` failed with this exact signature (same tests, same assertion line, same error body); only the v0.7.3 run passed.
- Reproduces on master both before (run 28674775656) and after (run 28689461523) the #507 capacity-cooldown merge, so it is not caused by recent coordinator changes.
- Also surfaced on PR #509 (run 28684913472) — confirmed unrelated to that branch's changes (`catalog_size_override.go` / registry tests are not on the failing path).
## Suspects to investigate
- Why the Swift testbed provider's WebSocket drops under concurrent request load (provider-side crash/timeout vs coordinator-side close).
- Single-provider testbed has no failover, so one disconnect fails the whole assertion — consider whether the test should tolerate reconnect-and-retry or assert on the disconnect itself.
Classification from CI investigation: pre-existing master flake, high confidence.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with integration_test.go around line 554 and reproduce the failures from integration.yml, including the cited master runs. Trace the Swift testbed provider WebSocket disconnect under concurrent load and whether the coordinator or provider closes it. Done means identifying the cause and making the concurrent tests stable without masking the provider_error condition.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, swift
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100