Result: opencode CLI + DeepSeek V4 Flash 0731 (agent1st protocol) — 384/391 (98.2%)

[results-v37-flash-0731-full391.json](https://github.com/user-attachments/files/30615536/results-v37-flash-0731-full391.json)

# Result: opencode CLI + DeepSeek V4 Flash 0731 (agent1st protocol) — 384/391 (98.2%)

## Result row for README

| Harness | Profile | Model | Result | % | Steps | Tokens |
| --- | --- | --- | ---: | ---: | ---: | ---: |
| opencode | — | DeepSeek V4 Flash 0731 (agent1st_v37.1) | 384/391 | 98.2% | — | — |

## Details

- **Harness:** `opencode` CLI (standalone, each task = fresh `opencode run` subprocess, no persistent server)
- **Agent:** `agent1st_v37.1-flash-0731` — Agent1st Protocol v37.1 (647-line system instruction, [agents.md](https://agents.md))
- **Model:** `opencode-go/deepseek-v4-flash` (DeepSeek V4 Flash 0731, 1M context)
- **Task set:** v0.16.0 (391 tasks, current `main`)
- **Config:** LSP disabled, MCP disabled, `--variant high` (reasoning_effort=high), `--dangerously-skip-permissions`
- **Concurrency:** 1 (no resource conflicts)
- **Per-task timeout:** 900s
- **OpenCode version:** v1.18.10
- **Run type:** single pass, first attempt, no reruns, no task-specific fixes

## Per-wave breakdown

| Wave | Tasks | Passed | % |
| --- | ---: | ---: | ---: |
| core (1–30) | 30 | 30 | 100.0% |
| extra (31–60) | 30 | 30 | 100.0% |
| more (61–100) | 40 | 40 | 100.0% |
| hard (101–150) | 50 | 49 | 98.0% |
| extreme (151–205) | 55 | 54 | 98.2% |
| diagnostic (206–221) | 16 | 15 | 93.8% |
| memory (222–253) | 32 | 30 | 93.8% |
| agentic (254–298) | 45 | 45 | 100.0% |
| VCS (299–313) | 15 | 15 | 100.0% |
| skills (314–330) | 17 | 17 | 100.0% |
| adversarial (331–351) | 21 | 21 | 100.0% |
| tbench-lite (352–371) | 20 | 18 | 90.0% |
| cli (372–391) | 20 | 20 | 100.0% |
| **Total** | **391** | **384** | **98.2%** |

## Failures (7)

| Task | Reason |
| --- | --- |
| `task_112_xlsx_sum_column` | Float format: `2655.0` vs expected int `2655` (K=5 follow-up: 5/5 PASS → flake) |
| `task_201_reconcile_vip_users` | Float format in JSON: `330.0` vs expected `330` (K=5: 4/5 → stochastic) |
| `task_210_tar_manifest_with_hashes` | CSV header row missing (K=5: 4/5 → stochastic) |
| `task_229_memory_update_contacts` | Email not written to MEMORY.md — **environment artifact** (see note below) |
| `task_242_memory_knowledge_update_job_change` | Email not written to MEMORY.md — **environment artifact** (see note below) |
| `task_357_request_latency` | CSV row order mismatch (K=5 follow-up: 5/5 PASS → flake) |
| `task_361_jsonl_cleaner` | JSON not compact (K=5 follow-up: 5/5 PASS → flake) |

**K=5 classification:** 3 of 7 failures are flakes (5/5 pass on retry), 2 are stochastic (~80%), 2 are stable.

> **Environment note on task_229/task_242:** these two stable failures are caused by the local `vv-opencode` plugin's secrets-redaction layer, which masks email addresses as placeholder tokens (`__VVOC_SECRET_EMAIL_...__`) inside the agent's tool output. The agent faithfully writes the masked placeholder into `MEMORY.md`; the verifier then cannot find the real email substring. In the same task attempts, `README.md` and `package.json` contain the real email — confirming the agent correctly extracted and propagated the value where the masking layer did not intercept. These are not instruction or model failures; they are a local environment artifact. Mitigation (plugin config exclusion or `--pure` run) is being prepared.

## Evidence

Full per-task result JSON attached:

- `results-v37-flash-0731-full391.json` (391 tasks, 384 passed, pass_rate=0.9821, task_set_version=0.16.0)

Summary: `{"total": 391, "passed": 384, "pass_rate": 0.9821}`

## Reproduction

```bash
cd /path/to/harness-bench-fast
OPENCODE_CONFIG=opencode-bench.json \
uv run python -m harness_bench run-cli \
--cli-command 'opencode run -m opencode-go/deepseek-v4-flash --agent A/agent1st_v37.1-flash-0731 --variant high --dangerously-skip-permissions' \
--concurrency 1 --timeout 900 \
--json-output results.json
Full agent definition: agent1st_v37.1-flash-0731.md (647 lines, included in the repo).
Comparison with existing rows
Row
Existing `deepagents
Existing `opencode
This submission
The same model (DeepSeek V4 Flash) jumps from 81.8% (deepagents stock) to 98.2% (opencode + Agent1st protocol) — demonstrating that agent-instruction quality matters as much as model capability. Two independent protocol iterations (v37 → v37.1 compact harness, −74 lines) kept the score stable while reducing instruction size by 10%.

results-v37-flash-0731-full391.json (https://github.com/user-attachments/files/your-upload-id/results-v37-flash-0731-full391.json)