boxlite-ai / boxlite-ai/boxlite
GUEST_READY_TIMEOUT is a hard-coded 30s with no override
- Dominant language
- Rust
- Stars
- 2.3k
- Forks
- 179
- Avg merge
- 23h 25m
- Merged PRs (30d)
- 121
Description
## Summary
`GUEST_READY_TIMEOUT` is a hard-coded 30 s with no environment or config override, so a box
that boots slowly on a contended host fails permanently with no available mitigation:
```
Box uPKTB5hOHtZf failed to start: timeout after 30s
Evidence at T+30s:
• shim_alive = true
• console_bytes = 0
• ready_socket_exists = true
• likely_cause = guest agent never wrote to console (init or vsock plumbing broken)
```
`src/boxlite/src/litebox/init/tasks/guest_connect.rs:110`:
```rust
const GUEST_READY_TIMEOUT: Duration = Duration::from_secs(30);
```
It has exactly two references — the definition and the call site at `:85`. There is no env var,
no `AdvancedOptions` field, and no per-box knob, so a caller on a loaded machine has no way to
raise it. The doc comment says it is "exposed as a constant so tests can call
`wait_for_guest_ready` with a short timeout", i.e. tests can shorten it but production callers
cannot lengthen it.
## How it showed up
Running the full Python SDK integration suite on macOS (`make test:integration:python`, which
the pre-push hook runs), `tests/test_tcp_filter.py::TestEdgeCases::test_multiple_allowed_hosts`
intermittently fails with the message above. It failed 3 times during full-suite runs.
## Reproduction attempts — what does and does not trigger it
I could not reproduce it in isolation. Recording the negative results so the next person does
not repeat them:
| Condition | Result |
|---|---|
| Standalone, idle machine (3 consecutive runs) | pass — 15.9 s, 15.6 s, 17.6 s |
| Standalone, 24 CPU spinners (load avg 20.6 / 18 cores) | pass — 28.4 s |
| Standalone, 40 CPU spinners + 4 `dd` I/O loops (load avg 52.9) | pass — 69.2 s |
| Whole `test_tcp_filter.py` file, 21 boxes back to back | pass — 21/21, 179 s |
| During / immediately after the **full** integration suite | **fails**, 3 occurrences |
Two things worth noting:
- Wall time degrades sharply with host load (15.9 s idle → 28.4 s → 69.2 s), so box startup is
clearly contention-sensitive; but pure CPU/IO pressure alone did not push the *guest-ready
handshake* past 30 s here.
- The failure signature is `console_bytes = 0` — the guest wrote *nothing*, which does not look
like ordinary slowness. Something about running the whole suite (hundreds of boxes) is the
trigger; I did not isolate what.
## Why the constant is worth fixing regardless
Whatever the underlying trigger turns out to be, a user or CI job that hits it today has no
escape hatch: they cannot raise the deadline to confirm "slow" versus "hung", and the same
30 s applies to a warm laptop and a heavily loaded CI box alike.
Suggested: make it overridable (env var and/or an `AdvancedOptions` field) with the current
30 s as the default, keeping the existing diagnostic block on expiry.
## Related
- #469 — the Node SDK hits the same class of failure under parallel test execution. That one is
a *vitest* 120 s hook timeout, i.e. a test-framework limit; this issue is about boxlite's own
internal deadline, which no caller can configure.
## Environment
- macOS 27.0.0, arm64, 18 cores
- boxlite 0.9.7, local (non-REST) runtime, `sdks/python` built from source
Contributor guide
Research direction
Start in src/boxlite/src/litebox/init/tasks/guest_connect.rs, reading the GUEST_READY_TIMEOUT definition and its call site around lines 85 and 110. Trace how AdvancedOptions and per-box configuration are passed, then run the relevant integration command; done means callers can raise the timeout while 30 seconds remains the default and expiry diagnostics are preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100