1jehuang / 1jehuang/jcode

Memory judge circuit breaker hides permanent faults: a 404 model-not-found is treated as a transient blip

Open
#800 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: no bug triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Problem

Split out from #798 as a follow-up, since the underlying weakness is independent of that specific bad model id.

In memory_rerank.rs, every sidecar error is funnelled through one path regardless of kind:

https://github.com/1jehuang/jcode/blob/master/crates/jcode-base/src/memory_rerank.rs#L292-L305

Err(e) => {
    record_failure_backoff(&e);
    crate::logging::event_rate_limited(
        crate::logging::LogLevel::Warn,
        "memory_consensus_judge_failed",
        Duration::from_secs(60),
        "Memory consensus judge failed; circuit breaker armed",
        vec![("error", e.to_string())],
    );
    None // transport error = no vote
}

A circuit breaker is the right response to a transient fault: the network blipped, back off, retry later. It is the wrong response to a permanent configuration fault. A 404 not_found_error naming a model that does not exist will fail identically on every future call, forever. Backing off just converts a hard error into an invisible one.

Because the judge is the only thing allowed to surface memory ("the judge is the only thing allowed to surface memory, so even one item must clear it"), the observable result is not an error but silently degraded recall. The user sees a feature that appears to work and quietly returns nothing.

In #798 this hid a permanently-broken model id for the entire lifetime of the sidecar feature. The only trace was a rate-limited WARN in a log file nobody reads unless already suspicious. That is a long time for a feature to be dead with no signal.

Expected behavior

  • Permanent faults (model not found, auth rejected, malformed request) are distinguished from transient ones (timeout, connection reset, 5xx, rate limit).
  • A permanent fault surfaces once, visibly, rather than arming a breaker that suppresses it.
  • Transient faults keep the current backoff behavior, which is correct for them.

Fix

Sketch, open to a different shape:

  1. Classify sidecar errors into transient vs permanent. HTTP 404 / 401 / 403 / 400 are permanent; timeouts, connection errors, 429 and 5xx are transient.
  2. Only record_failure_backoff for transient errors.
  3. For permanent errors, surface once to the user (startup notice or a one-shot status line) instead of silently disabling the feature. It is a misconfiguration the user can act on, unlike a network blip.

A cheap subset of the value: log permanent faults at ERROR rather than WARN, and exempt them from rate-limiting so the first occurrence always lands.

Likely affected files:

  • crates/jcode-base/src/memory_rerank.rs
  • crates/jcode-base/src/sidecar.rs (error typing, if classification lives with the transport)

Related: #798 (the bug this weakness concealed).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/jcode-base/src/memory_rerank.rs around the sidecar error handler at lines 292-305, then inspect crates/jcode-base/src/sidecar.rs for the existing error types. Trace how errors reach the user and how record_failure_backoff is triggered. Done means permanent and transient faults follow separate paths, with permanent faults surfaced visibly once and transient faults retaining backoff behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.