oceanbase / oceanbase/powercontext
feat(memory): add an optional Jev reranker with comparative evaluation
@Oxidaner is already working on this.
Since Sep 19, 2026.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 212
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 199
Description
Tracking issue: #1649
Feature description
Add an opt-in Jev implementation of MemoryReranker and evaluate it against the existing retrieval and LLM reranking paths. The intended benefit is to retain useful project evidence at lower latency and cost, without reducing downstream task success.
Priority: P0 — first integration experiment.
Problem and proposed solution
A coding agent can retrieve many entries about a database issue while missing the entry that explains the relevant compatibility constraint. Topic similarity alone does not establish usefulness for the current task.
PowerContext already exposes MemoryReranker and accepts an injected reranker. Implement a bounded adapter that sends the query and eligible candidate text to Jev, asks explicit relevance questions, and converts the resulting scores into validated original candidate ranks. Jev returns typed decisions; replacing the generation model name in the existing listwise generator is not sufficient.
Keep candidate identity, exact citations, authorization, lifecycle eligibility, and final UTF-8 budget enforcement in PowerContext. Start with Memory only. Pin the Jev model version and question-policy version; report usage, latency, and fallback without logging private candidate text by default. Configure a total deadline that includes any SDK retries. On timeout, rate limiting, or unusable results, preserve the original coarse order and report the fallback.
Use a bounded batch or bounded concurrent pair requests based on measured accuracy and latency. Do not assume that packing every candidate into shared state is always better.
Acceptance criteria
- The adapter implements the existing
MemoryRerankercontract and is disabled unless explicitly configured. Normal installations do not require Jev credentials. - Results contain only unique original candidate ranks within the requested limit; exact evidence identity is preserved through final preparation.
- Failure and deadline cases return the existing candidate order through a documented fallback. PreparedContext remains read-only and within its existing byte budget.
- Compare no reranking, the current LLM reranker, and Jev using the same candidate pools, query set, downstream model, and context budget. Include a suitable dedicated reranker baseline if one is available.
- Cover English and Chinese queries, lexical distractors, complementary evidence, negation, date/version conditions, and queries with no useful candidate.
- Report relevant-evidence recall, ranking quality, actual injected evidence/bytes, downstream task success, p50/p95 latency, billed input, retries, and total request cost.
- Validate integration behavior on SQLite and OceanBase, including fallback and authorization boundaries; publish reproducible artifacts and a recommendation to adopt, revise, or stop. Define the quality-regression tolerance before the held-out run.
Alternatives considered
Retaining the existing LLM reranker, using a dedicated reranking model, and improving deterministic retrieval are valid baselines. A lower provider price alone is not sufficient to select Jev.
Additional context
The model-free recall sufficiency gate in #1556 / #1596 is already on master. This issue adds a candidate-ordering implementation; it does not replace the gate or expand the retrieval pool. Keep expansion disabled with reranking unless the deployment explicitly enables and budgets that combination. Broader Memory quality/lifecycle policy remains in #1590; the evaluation framework remains in #1422.
- Current reranker interface
- Runtime injection point
- TypeSafe reranking cookbook
- Model limits and versioning
Prepared with AI assistance. This is proposed work; no Jev performance result has been reproduced in PowerContext.
Related Jev evaluations: #1644 assesses evidence coverage; #1646 demonstrates project-aware decisions independently of the reranking experiment.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.