Avijit-Kumar-GIT / Avijit-Kumar-GIT/fella
Fail fast on a hung provider request (client-side timeout)
- Dominant language
- Rust
- Stars
- 0
- Forks
- 0
- Avg merge
- 5h 19m
- Merged PRs (30d)
- 72
Description
Surfaced by `nemotron-3.5-lightning` in the ladder run (#63). Its `bare` pass was 100% clean; its `fella` pass ERR'd ~50% of tool-calling requests — each stalling **18–49 s** before failing. Two separate runs, same result: the OpenRouter/NVIDIA endpoint for that model can't sustain multi-round tool-calling payloads.
**Problem.** A slow or wedged upstream call ties up the run for tens of seconds per attempt with no ceiling. A user on a flaky endpoint sees Fella "thinking" for a minute, then an error.
**Direction.** A client-side per-request timeout on the LLM HTTP call (`engine/llm.rs` / the `reqwest` client), tuned so a normal slow response still lands but a wedged one fails in a few seconds and surfaces a clear "the model endpoint isn't responding" message. Consider a shorter timeout for the tool-calling turns (bigger payloads, where this bit) vs the final answer.
Low priority — it's a robustness nicety, not a correctness bug, and it's the endpoint's fault. But it makes a bad provider degrade gracefully instead of hanging.
Contributor guide
Research direction
Start in engine/llm.rs by locating the reqwest client and the LLM HTTP call. Review how tool-calling turns and final answers are sent, then define timeout behavior that allows normal slow responses while failing stalled requests with a clear endpoint-not-responding message. Done means wedged provider calls stop within the chosen limit and surface that message instead of hanging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend, networking
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100