Avijit-Kumar-GIT / Avijit-Kumar-GIT/fella

Fail fast on a hung provider request (client-side timeout)

Open
#69 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
0
Forks
0
Avg merge
5h 19m
Merged PRs (30d)
72

Description

Surfaced by `nemotron-3.5-lightning` in the ladder run (#63). Its `bare` pass was 100% clean; its `fella` pass ERR'd ~50% of tool-calling requests — each stalling **18–49 s** before failing. Two separate runs, same result: the OpenRouter/NVIDIA endpoint for that model can't sustain multi-round tool-calling payloads.

**Problem.** A slow or wedged upstream call ties up the run for tens of seconds per attempt with no ceiling. A user on a flaky endpoint sees Fella "thinking" for a minute, then an error.

**Direction.** A client-side per-request timeout on the LLM HTTP call (`engine/llm.rs` / the `reqwest` client), tuned so a normal slow response still lands but a wedged one fails in a few seconds and surfaces a clear "the model endpoint isn't responding" message. Consider a shorter timeout for the tool-calling turns (bigger payloads, where this bit) vs the final answer.

Low priority — it's a robustness nicety, not a correctness bug, and it's the endpoint's fault. But it makes a bad provider degrade gracefully instead of hanging.

Contributor guide

Open the contributing guide

Research direction

Start in engine/llm.rs by locating the reqwest client and the LLM HTTP call. Review how tool-calling turns and final answers are sent, then define timeout behavior that allows normal slow responses while failing stalled requests with a clear endpoint-not-responding message. Done means wedged provider calls stop within the chosen limit and surface that message instead of hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend, networking
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.