anthropics / anthropics/skills

Proposal: add skill-grader — a quality-rubric evaluation skill

Open
#1,555 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
176k
Forks
20.9k
Avg merge
7h 21m
Merged PRs (30d)
5

Description

**Summary**

I'd like to propose adding a new skill, `skill-grader`, which evaluates other Agent Skills against a production-grade quality rubric: 3 hard gates (YAML frontmatter, SKILL.md < 500 lines, description ≤ 1024 chars) + six-dimension weighted scoring (trigger design / workflow discipline / structure / output contract / guardrails / evaluation loop), producing evidence-based reports.

**Rationale**

This repo currently covers creation (`skill-creator`), document generation (`docx`/`pdf`/`pptx`/`xlsx`), and domain examples — but has no skill that *evaluates* skill quality. A grading skill completes the loop: create → evaluate → improve. It also operationalizes the guidance in the spec (progressive disclosure, optimized descriptions, evaluation).

**Implementation**

- Self-contained: SKILL.md (~180 lines) + examples/score_evals.md (3 self-test cases)
- No external dependencies; rubric embedded inline
- MIT licensed (can relicense to Apache-2.0 to match this repo if preferred)
- Reference implementation: https://github.com/yt2102/skill-grader

**Questions**

1. Does this repo accept community-contributed skills? (No CONTRIBUTING.md found.)
2. If yes, any requirements for submission (license, tests, spec compliance)?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the reference implementation at https://github.com/yt2102/skill-grader and the repository's existing skill layout. Confirm how community skills, licensing, and self-tests are handled, then add SKILL.md and examples/score_evals.md with the proposed three self-test cases. Done means the skill is self-contained, follows the repository's accepted requirements, and its evaluation rubric is exercised by the examples.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.