[Eval] offTargetExplored counts a nonexistent file under the right directory as a hit

What

offTargetExplored scores an exploration step by directory prefix only:

!e.path.includes('*') && !task.targetDirs.some((d) => e.path.startsWith(d))

A read_file on a path that does not exist, but which happens to sit under a declared target directory, is counted as on-target.

Observed

deep-query-route-source, same arm, two runs:

src/app/router.ts does not exist — the real file is src/app/routes/routeTable.ts. Run B scored offTarget: 0 because src/app/router.ts.startsWith(src/app), and then answered "I have only the directory scan, no file contents", because the read returned nothing.

So a hallucinated filename inside the correct directory is currently indistinguishable from a correct read. That is precisely the failure mode navigation is supposed to prevent — the model guessing router.ts when the repository says routes/routeTable.ts — and the metric is blind to it.

Why it matters for the ablation

This is the metric the filesense ablation reports as its primary signal. If a wrong-but-well-located guess scores the same as a correct read, the metric under-counts exactly the errors navigation should remove, biasing the comparison toward the null.

Suggested direction

Score against the step's outcome, not only its path. step_completed versus step_failed is already collected; a read_file that failed, or returned no content, should count as a miss regardless of directory. Distinguishing three buckets — on-target hit / on-target miss / off-target — would be more honest than the current binary, and the data to do it is already in the record.

Also worth noting: run-to-run variance

The same task on the same arm flipped PASS → FAIL between runs (39s / 1 exploration step versus 80s / 3 steps). Any conclusion drawn from a single run per arm cannot separate a small effect from this variance. Repeated runs, or a metric less binary than pass/fail, are needed before a difference is claimed.

Start at the implementation of offTargetExplored and trace how step_completed, failed read_file steps, and empty results are recorded. Update the metric to distinguish on-target hits, on-target misses, and off-target steps, then verify the example where src/app/router.ts does not exist is counted as a miss and account for repeated-run variance in the evaluation.