Interop: mapping PatchEval-Verified dataset + evaluation output to a portable TestCase/ResultSet format

Hi maintainers,

I maintain EvalPort (https://github.com/adhabnr-ux/evalport), an open interchange schema (`TestCase` / `Grader` / `Result` / `ResultSet` / `GraderResult`) plus Python (`evalport-sdk`) and TypeScript SDKs for making eval datasets and run results portable across harnesses.

I read `patcheval/datasets/patcheval_verified.json` and `patcheval/evaluation/run_evaluation.py` directly (not just the README) and think PatchEval-Verified is a clean fit for a standalone converter — similar in shape to the DeepEval adapter we already ship (`adapters/deepeval-openeval-adapter` — a real, tested package with a full field-mapping table, not a stub).

**Field mapping**

Dataset row → `TestCase`:
- `cve_id` → `TestCase.id`
- `cve_description` → `TestCase.input`
- `vul_func` → `TestCase.context` (reference-only, matching your own note that `fix_func`/`patch_url` shouldn't be used as a repair source, so I'd keep `fix_func` out of the agent-visible fields entirely)
- `repo`, `cwe_info`, `patch_url`, `programming_language`, `image_url` → `TestCase.metadata`

Evaluation output → `ResultSet`. `process_patch()` already collects exactly `(cve, language, validation_type, image_name, is_success, is_error)` per case before folding it into `summary.json`:
- each tuple → one `Result(test_case_id=cve, passed=is_success, ...)`
- the Docker/PoC check itself → one `GraderResult(grader_id="poc_validation", type="code", score=1.0/0.0, passed=is_success, reason=validation_type)` — your four `validation_type` values (`Repair Success` / `apply_fail` / `compilation_fail` / `validation_fail`) map directly onto `GraderResult.reason`
- `summary.json`'s per-language pass-rate breakdown → `ResultSet.summary`

**Sketch** (adapter-side only, doesn't touch the Docker/PoC logic):

```python
from openeval.types import TestCase, Result, GraderResult

def dataset_row_to_testcase(row: dict) -> TestCase:
return TestCase(
id=row["cve_id"],
input=row["cve_description"],
graders=["poc_validation"],
context=[row.get("vul_func", "")],
metadata={
"repo": row["repo"],
"cwe_info": row.get("cwe_info"),
"patch_url": row.get("patch_url"),
"programming_language": row["programming_language"],
"image_url": row["image_url"],
},
)

def eval_result_to_openeval(cve, validation_type, is_success, run_poc_msg) -> Result:
return Result(
test_case_id=cve,
passed=is_success,
grader_results=[GraderResult(
grader_id="poc_validation",
type="code",
score=1.0 if is_success else 0.0,
passed=is_success,
reason=validation_type,
metadata={"run_poc_msg": run_poc_msg[-2000:] if run_poc_msg else None},
)],
)
```

I'd be glad to build this as a standalone `patcheval-openeval-adapter` pip package (zero footprint here, same pattern as the DeepEval one) if that's useful — or, if you'd rather it live in-repo, I could open a PR adding an optional `--openeval-output` flag to `run_evaluation.py` that writes a `ResultSet` alongside `summary.json`, without changing the existing pass/fail logic or report format. Happy to go either way, or drop this if it's not a direction you're interested in.

— Sahi, independent contributor (not affiliated with this project)

Start by reading patcheval/datasets/patcheval_verified.json and patcheval/evaluation/run_evaluation.py, especially process_patch(), then compare the existing adapters/deepeval-openeval-adapter package. Confirm whether the work belongs in a standalone adapter or behind an optional --openeval-output flag. Done means the stated dataset and evaluation mappings produce TestCase/ResultSet output without changing existing pass/fail logic or summary.json.

Written by the indexing model from the issue text.