InseeFrLab / InseeFrLab/auto-tuning-vllm
CLI hangs after trial completes when launched from K8s UI terminal and session is closed
- Dominant language
- Python
- Stars
- 6
- Forks
- 0
- Avg merge
- 34m
- Merged PRs (30d)
- 2
Description
## Summary
When `auto-tune-vllm optimize` is started from an interactive Kubernetes UI terminal (exec) and the user closes the tab / disconnects, the optimization can appear to hang after a trial finishes:
- The trial worker completes successfully (benchmark + vLLM cleanup OK in trial logs)
- The main `auto-tune-vllm` process stays alive but stops making progress
- Optuna is not updated (`study.tell()` never runs or the main loop never continues)
- No subsequent trial is submitted
The same study configuration works reliably when:
- The user stays connected to the terminal, or
- The command is run as a Kubernetes Job (non-interactive stdout)
This was initially misdiagnosed as SQLite/NFS locking or trace-replay-specific behaviour, but reproduces across benchmark types when the terminal session is detached.
## Root cause
The CLI uses Rich live UI (`Progress` spinner + `RichHandler`) writing continuously to stdout.
When the K8s web terminal session ends:
- stdout becomes a broken pipe or an unread pipe
- The main thread blocks on the next console write (Rich spinner refresh or `logger.info()` via RichHandler)
- The worker thread (trial execution in `ThreadPoolExecutor`) is independent and can finish normally
- The orchestration loop in `StudyController` never collects the result and never calls `study.tell()`
Relevant code: `auto_tune_vllm/cli/main.py` — `Progress(SpinnerColumn(), ...)` + `RichHandler`.
## Workarounds (today)
- Run as a Kubernetes Job instead of a detached exec session
- Launch with `nohup` and redirect stdout/stderr to a file should work as well
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in auto_tune_vllm/cli/main.py, reading the Rich Progress spinner and RichHandler setup and how the CLI starts StudyController. Reproduce the command with stdout detached or closed, then trace whether console writes block the orchestration loop. Done means a detached session still lets the completed trial be collected, study.tell() run, and the next trial start.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- cli, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100