DeepSeek V4 Flash scores 60/89 in OpenCode versus DeepSeek's reported 74/89

Description

OpenCode 1.18.7 passed 60 of 89 Terminal-Bench 2.1 tasks with deepseek/deepseek-v4-flash. The score was 67.42%. DeepSeek reports 82.7% for V4-Flash-0731 on the same benchmark. The gap is 15.28 percentage points.

This is not a strict agent-only comparison. DeepSeek used DeepSeek Harness minimal mode at max effort. This OpenCode run did not set a variant, so the API used its documented default high effort. Please help confirm whether OpenCode sends and preserves all DeepSeek controls and reasoning data correctly.

Run contract:

• Run date: 2026-08-11 to 2026-08-13
• Dataset: terminal-bench/terminal-bench-2-1
• Dataset digest: sha256:7d7bdc1cbedad549fc1140404bd4dc45e5fd0ea7c4186773687d177ad3a0699a
• Harbor: 0.20.0
• OpenCode: 1.18.7
• Model: deepseek/deepseek-v4-flash
• Environment: Docker
• Attempts: 1 per task
• Retries: 0
• Result: 60 pass, 29 fail, 67.42%
• Evidence: 89 of 89 final verdicts are attached

Please check these points:

1. Does --variant max send reasoning_effort: "max" to DeepSeek in headless runs?
2. Does OpenCode return reasoning_content after every tool call, as DeepSeek requires?
3. Does the Chat Completions path behave differently from the Responses API path?
4. Are context limits, output limits, compaction, and timeouts reducing task success?
5. Does OpenCode report cache and reasoning usage correctly?

DeepSeek source: <https://api-docs.deepseek.com/updates/#date-2026-07-31

Plugins

No response

OpenCode version

1.18.7

Steps to reproduce

1. Use Harbor 0.20.0 with Docker.
2. Use dataset terminal-bench/terminal-bench-2-1@sha256:7d7bdc1cbedad549fc1140404bd4dc45e5fd0ea7c4186773687d177ad3a0699a.
3. Use Harbor's built-in opencode agent with OpenCode 1.18.7.
4. Use model deepseek/deepseek-v4-flash.
5. Do not set an OpenCode variant. This matches the observed run and uses DeepSeek's default high effort.
6. Run the 89 attached task IDs once. Use 18 concurrent tasks and agents, setup timeout multiplier 5, build timeout multiplier 2, agent timeout multiplier 20, and zero retries.
7. Count a pass only when reward is 1.0 and no exception exists. Count a timeout as a failure.
8. Compare the result with a second run that uses --variant max.

Observed result: 60/89, or 67.42%.

Expected result: OpenCode should expose a documented, working configuration for DeepSeek's max effort. It should preserve reasoning data across tool calls. A max-effort rerun should explain whether the 15.28-point gap comes from effort, harness behavior, or another integration problem.

Screenshot and/or share link

matched-plain-60-of-89.csv
matched-plain-60-of-89.json

Operating System

Docker via Harbor 0.20.0 on AWS

Terminal

Harbor 0.20.0 built-in OpenCode agent, non-interactive.