LocalEvalService discards per-invocation results for any metric reporting EvalStatus.NOT_EVALUATED -- is there an intended shape for non-pass/fail (measurement) metrics?

## 🔴 Required Information

**Describe the Bug:**
`LocalEvalService._evaluate_metric_for_eval_case` (local_eval_service.py:428-436)
discards a metric's real per-invocation `PerInvocationResult` (score, rubric_scores)
and substitutes an empty one whenever `EvaluationResult.overall_eval_status ==
EvalStatus.NOT_EVALUATED`:

```python
invocation_result = (
evaluation_result.per_invocation_results[idx]
if evaluation_result.overall_eval_status != EvalStatus.NOT_EVALUATED
else PerInvocationResult(actual_invocation=invocation.actual_invocation)
)
```

`NOT_EVALUATED` is also the *default* value of `EvaluationResult.overall_eval_status`
and `PerInvocationResult.eval_status` (confirmed via `model_fields` — both default to
`EvalStatus.NOT_EVALUATED`), so this fires for any custom `Evaluator` that reports a
real per-invocation measurement without opting into ADK's `score >= threshold ->
PASSED` pass/fail convention — which is the natural, low-friction thing to do for a
metric that isn't pass/fail by nature (a cost, a token count, a latency).

Downstream, `AgentEvaluator._process_metrics_and_get_failures` reads *only* from
these now-nulled per-invocation results to decide whether to raise. Since a metric
that never reports `PASSED` can never produce a non-empty `scores` list there, it
unconditionally raises `AssertionError` for **any** real value the metric computed —
no threshold configuration avoids this, because the per-invocation score was already
discarded before the threshold comparison happens.

I ran into this building a third-party ADK metric that reports real per-invocation
dollar cost (lower-is-better, so it deliberately never participates in ADK's
higher-is-better pass/fail gate — see "Additional Context" below). It surfaced two
separate effects:
1. `AgentEvaluator.evaluate()` always raises, regardless of the actual computed
value or the configured threshold.
2. `adk eval` (CLI) doesn't raise, but the per-invocation table and the persisted
`eval_history/*.evalset_result.json` both show `score: null` /
`rubric_scores: null` for every invocation. Only one coarse, un-persisted
console line (the aggregate `Metric: ... Score: X`) carries the real number —
nothing about *why* a score is present/absent, or any per-call breakdown, survives
anywhere a user can read after the run.

**Steps to Reproduce:**
Minimal, self-contained repro below (no third-party packages) — a toy metric that
reports a real per-invocation number (42.0) via a custom `Evaluator`, registered the
documented way via `DEFAULT_METRIC_EVALUATOR_REGISTRY.register_evaluator`, deliberately
leaving `eval_status` at its default (`NOT_EVALUATED`, since the metric isn't pass/fail).

`my_agent/agent.py`:
```python
from __future__ import annotations
from collections.abc import AsyncGenerator
from google.adk.agents.llm_agent import LlmAgent
from google.adk.evaluation.eval_case import ConversationScenario, Invocation
from google.adk.evaluation.eval_metrics import EvalMetric, Interval, MetricInfo, MetricValueInfo
from google.adk.evaluation.evaluator import EvaluationResult, Evaluator, PerInvocationResult
from google.adk.evaluation.eval_rubrics import RubricScore
from google.adk.evaluation.metric_evaluator_registry import DEFAULT_METRIC_EVALUATOR_REGISTRY
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_request import LlmRequest
from google.adk.models.llm_response import LlmResponse
from google.genai import types as genai_types

METRIC_NAME = "toy_measurement"

class ToyMeasurementEvaluator(Evaluator):
"""Reports a real per-invocation number, but is not pass/fail (e.g. a cost)."""
def __init__(self, *, eval_metric: EvalMetric) -> None:
self._eval_metric = eval_metric

def evaluate_invocations(self, actual_invocations, expected_invocations=None,
conversation_scenario=None) -> EvaluationResult:
per_invocation_results = [
PerInvocationResult(
actual_invocation=inv, expected_invocation=None, score=42.0,
rubric_scores=[RubricScore(rubric_id="toy", score=42.0,
rationale="the real breakdown")],
)
for inv in actual_invocations
]
return EvaluationResult(
overall_score=sum(r.score for r in per_invocation_results),
per_invocation_results=per_invocation_results,
# overall_eval_status intentionally left unset -> defaults to
# NOT_EVALUATED, which is correct: this metric is not pass/fail.
)

DEFAULT_METRIC_EVALUATOR_REGISTRY.register_evaluator(
metric_info=MetricInfo(
metric_name=METRIC_NAME,
description="Toy non-pass/fail measurement metric.",
metric_value_info=MetricValueInfo(
interval=Interval(min_value=0.0, max_value=1_000_000.0, open_at_max=True)),
),
evaluator=ToyMeasurementEvaluator,
)

class _FakeLlm(BaseLlm):
model: str = "fake-model"
@classmethod
def supported_models(cls) -> list[str]:
return ["fake-model"]
async def generate_content_async(self, llm_request: LlmRequest, stream: bool = False):
yield LlmResponse(content=genai_types.Content(
parts=[genai_types.Part(text="ok")], role="model"))

root_agent = LlmAgent(name="toy_agent", model=_FakeLlm(), instruction="Answer briefly.")
```

`my_agent/__init__.py`:
```python
from . import agent
```

Driver:
```python
import asyncio, json
from pathlib import Path
from google.adk.evaluation.eval_case import EvalCase, Invocation
from google.adk.evaluation.eval_set import EvalSet
from google.genai import types as genai_types

eval_case = EvalCase(eval_id="case_1", conversation=[
Invocation(user_content=genai_types.Content(parts=[genai_types.Part(text="hi")], role="user"))
])
Path("eval_set.json").write_text(EvalSet(eval_set_id="repro_set", eval_cases=[eval_case]).model_dump_json())
Path("test_config.json").write_text(json.dumps({"criteria": {"toy_measurement": 999999.0}}))

async def main():
from google.adk.evaluation.agent_evaluator import AgentEvaluator
await AgentEvaluator.evaluate(
agent_module="my_agent",
eval_dataset_file_path_or_dir="eval_set.json",
num_runs=1, print_detailed_results=True,
)

asyncio.run(main())
```

1. Install `google-adk[eval]==2.6.3`.
2. Run the driver script above (`agent_module_file_path` layout: `my_agent/__init__.py` + `my_agent/agent.py`, matching the standard convention).
3. Observe the raised `AssertionError`.

**Expected Behavior:**
A metric that computes a real per-invocation value (42.0 in the repro) and
deliberately reports `NOT_EVALUATED` because it isn't pass/fail should be able to
surface that value through `AgentEvaluator.evaluate()`/`adk eval` — at minimum
without crashing, and ideally with the per-invocation score and rationale intact
in both the printed table and the persisted eval-history JSON.

**Observed Behavior:**
```
Summary: `EvalStatus.NOT_EVALUATED` for Metric: `toy_measurement`. Expected threshold: `999999.0`, actual value: `None`.
+----+--------------------------+---------+-------------+----------+---------------------+-------------------+-----------------------+---------------------+
| | eval_status | score | threshold | prompt | expected_response | actual_response | expected_tool_calls | actual_tool_calls |
+====+==========================+=========+=============+==========+=====================+===================+=======================+=====================+
| 0 | EvalStatus.NOT_EVALUATED | | 999999 | hi | | ok | | |
+----+--------------------------+---------+-------------+----------+---------------------+-------------------+-----------------------+---------------------+
RESULT: evaluate() raised AssertionError:
Following are all the test failures.
toy_measurement for None Failed. Expected 999999.0, but got None.
```
The metric's `evaluate_invocations()` was called and did return `score=42.0` per
invocation (verifiable by instrumenting `ToyMeasurementEvaluator.evaluate_invocations`
directly) — the value is computed correctly and then discarded before it reaches any
output.

**Environment Details:**
- ADK Library Version: `google-adk==2.6.3`
- Desktop OS: Windows 11
- Python Version: 3.13.5

**Model Information:**
- Are you using LiteLLM: No
- Which model is being used: N/A (repro uses a fake `BaseLlm`, no real model call needed)

---

## 🟡 Optional Information

**Regression:**
Not a regression as far as I can tell — `local_eval_service.py:428-436`'s
NOT_EVALUATED branch appears intentional (it's guarding against evaluators that
return zero per-invocation results), it just also catches evaluators that return a
full, real per-invocation result set while legitimately reporting NOT_EVALUATED as
their permanent status.

**Additional Context — the underlying design question:**
This isn't only a bug report — I'd like to understand the intended contract here.
As far as I can tell, ADK's `Evaluator`/`EvalStatus` model has no way to express
"this metric measures something real per invocation, but that measurement isn't
pass/fail" — `EvalStatus` is `PASSED | FAILED | NOT_EVALUATED`, and
`NOT_EVALUATED` is treated by `LocalEvalService`/`AgentEvaluator` as "nothing to
report," not as "reported, deliberately not gated." Silently negating a
lower-is-better score to force a "pass" would misrepresent it to anyone reading the
result, so `NOT_EVALUATED` is the only status that doesn't lie — but that value turns
out to mean "discard everything" a few layers down, not "measured, not gated."

Is a non-pass/fail "measurement" metric (cost, token count, latency, or similar)
something ADK's `Evaluator` contract is meant to support today? If so, what's the
recommended shape for it — a distinct `EvalStatus` variant, a flag on `EvalMetric`/
`MetricInfo` that opts a metric out of the threshold-gating and per-invocation-nulling
behavior, something else? I'm happy to be pointed at existing conventions I missed,
or to help prototype once there's a direction — I just don't want to guess at API
design that maintainers would need to carry.

**How often has this issue occurred?:**
Always (100%) — deterministic given the repro above.