Skills evaluation and curation
Open
@tmckayus is already working on this.
Since Jul 8, 2026.
Agentic
awaiting response
Task
- Dominant language
- Cuda
- Stars
- 1k
- Forks
- 233
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 95
Description
Grow eval coverage per skill, add GPU-based eval, and add/remove skills based on eval results.
Already landed (post-26.06):
- Evals added: #1398, #1418, #1417, #1399, #1337, #1327, #1324, #1325, #1302/#1308, #1460 (plus benchmark-detail updates #1331–#1335)
- Curation/consolidation: consolidate 3 API skills (#1489), remove redundant concept-only skills (#1487), namespace refactor sweep (#1475–#1480), catalog onboarding (#1310/#1343/#1344/#1345), skill renames (#1328–#1330)
Remaining (no PRs yet):
- GPU-based evaluation for APIs — S
- Finalize NVCarps integration — XS
- Ongoing add/remove of skills from eval results — S
Owner: Ram · Release: 26.08
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.