Owner

TIGER-AI-Lab

10 indexed repositories · View on GitHub

verl-tool
A version of verl to support diverse tool use [TMLR 2026]

Python · 1041 stars
ClawBench
Open-source benchmark for browser AI agents on daily tasks.

Python · 795 stars
VLM2Vec
This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]

Python · 684 stars
ScholarCopilot
ScholarCopilot: Training Large Language Models for Academic Writing with Accurate Citations [COLM 2025]

Python · 252 stars
EditReward
EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing [ICLR 2026]

Python · 160 stars
Context-Forcing
Context Forcing: Consistent Autoregressive Video Generation with Long Context [ICML26]

Python · 119 stars
VideoScore2
Automatic Metric for Evaluating Generated Videos

Python · 58 stars
RewardHarness
Self-evolving agentic reward framework for image-editing evaluation — 47.4% on EditReward-Bench from only 100 preference demos, no reward...

Python · 54 stars
StructEval
Evaluating LLMs' abilities to generate structural output [TMLR2025]

Python · 23 stars
FIM-Midtraining

Python · 20 stars

Beginner friendly Everything

Open All

Proposal: isolated environmental-observation-retrieval suite with 12 distinct tasks Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

TIGER-AI-Lab/ClawBench#375 · 2 comments · Opened Sep 14, 2026
publicity: audit existing outreach PRs and make featured-in claims traceable Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 52/100

TIGER-AI-Lab/ClawBench#372 · Opened Sep 13, 2026
publicity: prepare a reproducible next-release demo and bilingual announcement kit Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 55/100

TIGER-AI-Lab/ClawBench#371 · Opened Sep 13, 2026
publicity: turn the model leaderboard call-to-action into an evidence-based submission guide Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 72/100

TIGER-AI-Lab/ClawBench#370 · Opened Sep 13, 2026
Next release: data safety, scoring correctness, recovery and package validation checklist Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

TIGER-AI-Lab/ClawBench#366 · 3 comments · Opened Sep 13, 2026
[P2] CI: runtime-server changes select zero downstream harness image builds Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 74/100

TIGER-AI-Lab/ClawBench#365 · Opened Sep 13, 2026
[P2] rescore: truncated judge cache is permanently skipped instead of retried Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

TIGER-AI-Lab/ClawBench#364 · Opened Sep 13, 2026
[P1] rescore: stale judge caches are silently attributed to a newly requested judge model Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 68/100

TIGER-AI-Lab/ClawBench#363 · Opened Sep 13, 2026
[P1] reproduce: validate sample coverage and selected rubrics before reporting PASS/FAIL Open

bug

TIGER-AI-Lab/ClawBench#362 · 1 comment · 1 assignee · Opened Sep 13, 2026
Harbor upstream: agree live-site oracle evidence and parity acceptance protocol Open

Difficulty 5/5 Over a week Newbie friendliness 32/100

TIGER-AI-Lab/ClawBench#359 · Opened Sep 13, 2026
Harbor: preserve verifier evidence and scoring when Kernel cleanup fails Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#358 · Opened Sep 13, 2026
Harbor: register browser MCP tools for the default local runtime Open

Difficulty 4/5 3-5 days Newbie friendliness 66/100

TIGER-AI-Lab/ClawBench#357 · Opened Sep 13, 2026
momentseeker数据不对 Open

Difficulty 4/5 3-5 days Newbie friendliness 15/100

TIGER-AI-Lab/VLM2Vec#212 · Opened Sep 8, 2026
video.yaml配置不对 Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

TIGER-AI-Lab/VLM2Vec#211 · Opened Sep 8, 2026
Priority issues to work on for contributors Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

TIGER-AI-Lab/ClawBench#342 · Opened Sep 8, 2026
memory.yaml配置文件 Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

TIGER-AI-Lab/VLM2Vec#210 · Opened Sep 8, 2026
Harbor: strict harbor/terminal-bench format compliance + e2b sandbox support Open

TIGER-AI-Lab/ClawBench#331 · 3 comments · 1 assignee · Opened Aug 31, 2026
Interop idea: map EvaluationExample/EvaluationResult onto EvalPort's TestCase/Result for portable benchmark output Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

TIGER-AI-Lab/RewardHarness#2 · Opened Aug 25, 2026
Proposal: isolated academia-registry-v0 suite for evidence-grounded registry research Open

question

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#317 · 2 comments · Opened Aug 20, 2026
feat(eval): support DeepSeek Harness and run a reproducible ClawBench benchmark Open

enhancement

TIGER-AI-Lab/ClawBench#309 · 1 comment · 1 assignee · Opened Aug 19, 2026
Stage-1 interceptor matching is duplicated between runtime-server and edgebench_judge, and the live copy has zero tests Open

Difficulty 4/5 3-5 days Newbie friendliness 64/100

TIGER-AI-Lab/ClawBench#301 · 1 comment · Opened Aug 18, 2026
judge_llm.py has diverged from judge.py (no Gemini support, wrong responses endpoint, 800-token cap) — collapse the duplication Open

Difficulty 3/5 1-2 days Newbie friendliness 70/100

TIGER-AI-Lab/ClawBench#300 · 1 comment · Opened Aug 18, 2026
batch --resume skips failed jobs (log-file heuristic) and overwrites batch-summary.json with 'skipped' rows Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

TIGER-AI-Lab/ClawBench#297 · 1 comment · Opened Aug 18, 2026
rescore: public reproducibility CLI defaults to a maintainer's home paths and silently no-ops elsewhere Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

TIGER-AI-Lab/ClawBench#296 · 1 comment · Opened Aug 18, 2026
Verify evals on Papers with Code Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

TIGER-AI-Lab/FIM-Midtraining#1 · Opened Aug 14, 2026
docs: expand repository topics for structured-output discovery Open

Difficulty 1/5 Under an hour Newbie friendliness 62/100

TIGER-AI-Lab/StructEval#6 · Opened Jul 26, 2026
Always report both Stage-1 (interception) and Stage-2 (judged); raw interception overcounts ~2× Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 62/100

TIGER-AI-Lab/ClawBench#243 · Opened Jul 2, 2026
aggregation_threshold Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

TIGER-AI-Lab/EditReward#29 · 1 reaction · Opened Jun 7, 2026
Data Annotation Format Issue Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

TIGER-AI-Lab/EditReward#28 · Opened Jun 4, 2026
feat: adapter — run AssistantBench (realistic time-consuming live-web tasks) under the ClawBench harness Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 48/100

TIGER-AI-Lab/ClawBench#188 · Opened May 20, 2026
feat: adapter — run Online-Mind2Web (live-web evolution of Mind2Web) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#189 · Opened May 20, 2026
feat: adapter — run WebVoyager (live-website, screenshot+LLM-judge browser agent) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#190 · Opened May 20, 2026
feat: adapter — run evolvent-ai/ClawMark (100 multi-day coworker-agent tasks, rule-based scoring) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 42/100

TIGER-AI-Lab/ClawBench#169 · Opened May 18, 2026
feat: adapter — run InternLM/WildClawBench (60 in-the-wild OpenClaw tasks) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#171 · Opened May 18, 2026
v3 training data Open

Difficulty 3/5 1-2 days Newbie friendliness 52/100

TIGER-AI-Lab/VLM2Vec#203 · Opened May 17, 2026
Release artifacts (models, dataset) on Hugging Face Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

TIGER-AI-Lab/RewardHarness#1 · Opened May 15, 2026
test(resilience): verify resume works — auto-restart on abort + alerting Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#160 · Opened May 13, 2026
EVALUATE ONLY Open

Difficulty 5/5 Over a week Newbie friendliness 10/100

TIGER-AI-Lab/ScholarCopilot#8 · Opened May 8, 2026
EVALUATION AND REDO Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

TIGER-AI-Lab/ScholarCopilot#7 · 1 comment · Opened May 8, 2026
NEWEST DRADT HARKIS Open

Difficulty 1/5 Under an hour Newbie friendliness 55/100

TIGER-AI-Lab/ScholarCopilot#6 · 1 comment · Opened May 8, 2026
FEAT: support steel browser Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#105 · Opened Apr 29, 2026
FEAT: support more browser runtimes Open

TIGER-AI-Lab/ClawBench#103 · 1 assignee · Opened Apr 29, 2026
Support running other tasks suits under ClawBench's infrastructure using adaptor layers. Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#72 · 1 comment · Opened Apr 15, 2026
评估支持使用vllm框架部署的模型吗 Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

TIGER-AI-Lab/VLM2Vec#200 · Opened Apr 3, 2026
About LORA Open

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

TIGER-AI-Lab/verl-tool#154 · Opened Apr 1, 2026
about Inference Result? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

TIGER-AI-Lab/VideoScore2#7 · Opened Mar 31, 2026
GPT-OSS-120B LoRA finetuning support? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

TIGER-AI-Lab/verl-tool#151 · Opened Mar 13, 2026
VD-Vidore-V2 evaluation metrics do not match the paper. Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

TIGER-AI-Lab/VLM2Vec#194 · 2 comments · Opened Mar 11, 2026
How to support eval Qwen2.5/3-omni Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

TIGER-AI-Lab/VLM2Vec#193 · Opened Mar 5, 2026
some question about the final reward score Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

TIGER-AI-Lab/EditReward#25 · 1 comment · Opened Mar 3, 2026

TIGER-AI-Lab

Get new issues in your inbox