[challenge]: Expanding ORBIT-Q with More Discriminative Quantum Programming Tasks

Released by

Shi-Xin Zhang, IOP-CAS

Contact email

shixinzhang@iphy.ac.cn

Method

Quantum Circuit Simulation

Challenge issue

ORBIT-Q currently contains 12 containerized quantum-programming tasks that evaluate whether autonomous coding agents can produce correct, framework-native, executable, and efficient scientific artifacts. This challenge asks participants to extend ORBIT-Q with additional hard tasks that preserve the same benchmark philosophy with discriminative power: the new tasks should be difficult for most current AI agents, while still being well-posed, automatically verifiable, and solvable by a careful human expert using TensorCircuit-NG.

The goal is not to create a separate benchmark. The goal is to add new ORBIT-Q-style tasks with the same structure: a clear task instruction, a
run_solution(config) interface, hidden/randomized functional checks, runtime measurement, static policy checks, and source-level audit for framework fidelity and shortcut avoidance.

The important requirement is that each task should expose a real quantum-programming bottleneck rather than a plain algorithm implementation.

参考 ORBIT-Q benchmark 的公众号介绍

Start by reviewing the existing ORBIT-Q benchmark at sxzgroup.github.io/ORBIT-Q/ and its 12 containerized quantum-programming tasks. Define an additional TensorCircuit-NG task around a real quantum-programming bottleneck, using the run_solution(config) interface. Done means the task has hidden or randomized functional checks, runtime measurement, static policy checks, and source-level audits for framework fidelity and shortcut avoidance.