Judge seam, [evaluate] table and a marker on untrusted tool results
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 2
- Forks
- 0
- Avg merge
- 1h
- Merged PRs (30d)
- 17
Description
What it does
Gives the engine a seam through which it can ask TypeSafe's Jev a question on its own, adds an [evaluate] table to ganja.toml that configures it, and uses it to screen the results of tools that bring outside text into a session (webfetch, websearch, and MCP servers named in the configuration), marking a result the judge scores as untrusted before the model reads it. A project's configuration may only narrow what is screened, never widen it, and the judge stays off under ganja serve.
Source
Outside beads: the plan the owner had put on hold, brought into the map as one idea during #47 (recorded in the amendment on #35). Today the only Jev surfaces are D564's evaluate tool, the ganja evaluate command (AGENTS.md:57) and the PreToolUse recipe (docs/recipes/typesafe-pretooluse-hook.sh).
File and line references in this issue are to main as of 75a7a92.
Against the three criteria
- Direction (README): The README does not mention Jev or
evaluate. The nearest passage is the isolation bullet's "blast radius of a poisoned prompt or a bad tool call" (README.md:23-27). - Today's terminal users: A user whose session reads web pages or MCP results gets a marker on text that tries to steer the model. It is also the base four other do ideas need.
- Evidence: The plan has been reviewed by a critic twice and revised to a third draft. It puts a measurement with the shipped
ganja evaluateahead of all code; nothing is measured on ganja yet.
Depends on
No other idea. It blocks #60, #88, #84 and #61, which all need the engine to ask Jev a question on its own. #62 reopens when this is built.
Verdict
- Verdict: do
- Size: M — one plan and one decision number, a few waves.
- Exception to the probe rule: yes. #36 says an idea that needs a measurement or a probe first is not do; this one is an explicit exception: its plan puts a measurement before any code, and the measurement is part of the work.
- Date: 2026-09-19
- Verdict ticket: #47
- The owner lifted the hold on this idea in #49: once the ideas are published, work may start from its plan.
Theme: #50
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing ganja evaluate command in AGENTS.md:57, D564's evaluate tool, and docs/recipes/typesafe-pretooluse-hook.sh; use the reviewed plan and its measurement as the starting point. Done means the engine can judge configured webfetch, websearch, and MCP results, mark untrusted text before model reads it, narrow screening through ganja.toml, and keep judging off under ganja serve.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100