open-compass/AgentCompass

[EMNLP 2026] AgentCompass is an extensible open-source evaluation infrastructure for systematically assessing LLM/VLM agent capabilities.

Stars: 135
Forks: 28
Open beginner issues: 0
Indexed issues: 3
Avg merge: 1d 10h
Merged PRs (30d): 37

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 17, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

3 open issues indexed

[Bug] 内置 ACTF 消费者无法一致解析非 OpenAI 工具调用 / Built-in ACTF consumers silently ignore non-OpenAI tool-call payloads Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

open-compass/AgentCompass#288 · 2 comments · Opened Aug 31, 2026
[Bug] reconstruct_run_result drops StepInfo.tools from persisted trajectories Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

open-compass/AgentCompass#276 · Opened Aug 20, 2026
[Bug] pass@k and avg@k collapse to attempt-1 accuracy in micro-weighted aggregation Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

open-compass/AgentCompass#273 · 1 comment · Opened Aug 20, 2026

A short digest of beginner-friendly GitHub issues.