githubnext / githubnext/gh-aw-workshop

[curriculum-eval] curriculum-scorecard: overall page scores and trend analysis

Closed
#3,439 0 comments 0 reactions 0 assignees View on GitHub
curriculum documentation quality
Dominant language
JavaScript
Stars
47
Forks
19
Avg merge
8h 54m
Merged PRs (30d)
50

Description

## Part Summary

| Part | Files | Mean Score | Std Dev |
|---|---|---|---|
| Part 1 — core path (lessons 00–14) | 55 | 6.25 / 10.0 | ±0.74 |
| Part 2 — advanced (lessons 15+) | 37 | 5.99 / 10.0 | ±0.37 |
| Overall corpus | 93 | 6.14 / 10.0 | ±0.62 |

## Part 1 Scores (lessons 00–14)

| File | Overall Score |
|---|---|
| workshop/side-quest-01-02-environment-reference.md | 4.93 / 10.0 |
| workshop/04-github-actions-intro.md | 5.39 / 10.0 |
| workshop/side-quest-11-06-anthropic-key.md | 5.39 / 10.0 |
| workshop/05-agentic-workflows-intro.md | 5.43 / 10.0 |
| workshop/14b-pr-reviewer-workflow.md | 5.45 / 10.0 |
| workshop/side-quest-05-03-two-file-structure.md | 5.51 / 10.0 |
| workshop/side-quest-11-01b-workflow-structure.md | 5.59 / 10.0 |
| workshop/side-quest-13-04-token-optimization.md | 5.61 / 10.0 |
| workshop/side-quest-11-08-frontmatter-tools-outputs.md | 5.64 / 10.0 |
| workshop/08-run-your-workflow.md | 5.67 / 10.0 |
| workshop/side-quest-11-04-annotated-workflow.md | 5.67 / 10.0 |
| workshop/05b-agentic-workflows-security.md | 5.75 / 10.0 |
| workshop/side-quest-06-03a-copilot-requests-permission.md | 5.75 / 10.0 |
| workshop/14-next-steps.md | 5.83 / 10.0 |
| workshop/side-quest-05-01-actions-power-user.md | 5.83 / 10.0 |
| workshop/side-quest-06-03-copilot-token.md | 5.83 / 10.0 |
| workshop/side-quest-10-02-jailbreak-brief.md | 5.85 / 10.0 |
| workshop/side-quest-11-05-event-triggers.md | 5.87 / 10.0 |
| workshop/side-quest-11-07-openai-key.md | 5.91 / 10.0 |
| workshop/side-quest-10-03-skill-dispatcher.md | 5.95 / 10.0 |
| workshop/side-quest-09-01f-debugging-checklist.md | 5.96 / 10.0 |
| workshop/08b-interpret-your-run.md | 5.99 / 10.0 |
| workshop/09-agentic-editing.md | 6.01 / 10.0 |
| workshop/side-quest-08-01-codespaces-actions-write.md | 6.01 / 10.0 |
| workshop/02a-setup-codespace.md | 6.09 / 10.0 |
| workshop/side-quest-10-01-agent-brief.md | 6.09 / 10.0 |
| workshop/side-quest-05-02-aw-deep-dive.md | 6.13 / 10.0 |
| workshop/side-quest-07d-billing-paths.md | 6.15 / 10.0 |
| workshop/side-quest-09-01-debug-output.md | 6.15 / 10.0 |
| workshop/side-quest-09-01a-pattern-long-plan-chain.md | 6.20 / 10.0 |
| workshop/05c-agentic-workflows-practice.md | 6.22 / 10.0 |
| workshop/side-quest-11-03-better-prompts.md | 6.24 / 10.0 |
| workshop/07-your-first-workflow.md | 6.25 / 10.0 |
| workshop/07d-confirm-model-access.md | 6.25 / 10.0 |
| workshop/side-quest-02-01-local-terminal.md | 6.25 / 10.0 |
| workshop/side-quest-01-01-terminal-basics.md | 6.26 / 10.0 |
| workshop/side-quest-09-01b-pattern-empty-results.md | 6.35 / 10.0 |
| workshop/side-quest-09-01c-pattern-safe-output-blocked.md | 6.35 / 10.0 |
| workshop/side-quest-09-01d-pattern-permission-denied.md | 6.37 / 10.0 |
| workshop/side-quest-09-01e-pattern-done-no-write.md | 6.39 / 10.0 |
| workshop/side-quest-06-01-install-troubleshooting.md | 6.41 / 10.0 |
| workshop/side-quest-06-03c-copilot-github-token-ui-only.md | 6.45 / 10.0 |
| workshop/side-quest-07-01-compile-workflow.md | 6.45 / 10.0 |
| workshop/side-quest-13-03-pr-checklist-pattern.md | 6.45 / 10.0 |
| workshop/side-quest-01-03-permission-errors.md | 6.53 / 10.0 |
| workshop/side-quest-06-04-install-local.md | 6.67 / 10.0 |
| workshop/side-quest-06-03b-copilot-github-token.md | 6.71 / 10.0 |
| workshop/side-quest-11-02-yaml-frontmatter.md | 6.88 / 10.0 |
| workshop/side-quest-11-01-frontmatter-deep-dive.md | 6.97 / 10.0 |
| workshop/side-quest-06-02-cca-codespace.md | 7.25 / 10.0 |
| workshop/side-quest-11-09-agent-session-phases.md | 7.69 / 10.0 |
| workshop/side-quest-13-01-schedule-expressions.md | 7.89 / 10.0 |
| workshop/side-quest-12-01-iterate-agent-output.md | 7.91 / 10.0 |
| workshop/side-quest-13-02-pr-summary-pattern.md | 8.39 / 10.0 |
| workshop/side-quest-13-01-pr-labeler-pattern.md | 8.63 / 10.0 |
## Part 2 Scores (lessons 15+)

| File | Overall Score |
|---|---|
| workshop/side-quest-17-07-repo-poisoning.md | 4.97 / 10.0 |
| workshop/15-conditional-logic.md | 5.53 / 10.0 |
| workshop/side-quest-20-01-memory-patterns.md | 5.57 / 10.0 |
| workshop/side-quest-22-01-failure-modes.md | 5.57 / 10.0 |
| workshop/17-add-mcp-tools.md | 5.61 / 10.0 |
| workshop/20-persistent-memory.md | 5.67 / 10.0 |
| workshop/16-connect-data-source.md | 5.71 / 10.0 |
| workshop/29-skills-and-domain-knowledge.md | 5.75 / 10.0 |
| workshop/side-quest-16-02-secrets-and-permissions.md | 5.77 / 10.0 |
| workshop/side-quest-17-02-security-architecture.md | 5.77 / 10.0 |
| workshop/side-quest-16-03-token-exfiltration.md | 5.79 / 10.0 |
| workshop/side-quest-17-03-prompt-injection.md | 5.81 / 10.0 |
| workshop/side-quest-23-01-ab-roundrobin.md | 5.83 / 10.0 |
| workshop/26-manage-costs-and-budgets.md | 5.85 / 10.0 |
| workshop/side-quest-17-01-mcp-concepts.md | 5.89 / 10.0 |
| workshop/side-quest-17-04-permission-escalation.md | 5.93 / 10.0 |
| workshop/side-quest-25-01-audit-reference.md | 5.93 / 10.0 |
| workshop/21-inline-sub-agents.md | 5.97 / 10.0 |
| workshop/side-quest-16-05-long-lived-credentials.md | 5.97 / 10.0 |
| workshop/side-quest-17-06-output-injection.md | 5.97 / 10.0 |
| workshop/side-quest-17-05-supply-chain-mcp.md | 6.01 / 10.0 |
| workshop/side-quest-15-01-expressions-and-contexts.md | 6.03 / 10.0 |
| workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md | 6.03 / 10.0 |
| workshop/side-quest-29-01-skill-injection-strategies.md | 6.09 / 10.0 |
| workshop/28-orchestrate-workflows.md | 6.13 / 10.0 |
| workshop/side-quest-24-01-runner-infrastructure.md | 6.13 / 10.0 |
| workshop/18-share-and-reuse.md | 6.15 / 10.0 |
| workshop/25-audit-and-observability.md | 6.15 / 10.0 |
| workshop/side-quest-26-01-forecast-costs.md | 6.20 / 10.0 |
| workshop/side-quest-15-02-chaining-conditions.md | 6.21 / 10.0 |
| workshop/22-error-handling-and-resilience.md | 6.35 / 10.0 |
| workshop/19-research-driven-training-node.md | 6.37 / 10.0 |
| workshop/24-self-hosted-runners.md | 6.45 / 10.0 |
| workshop/side-quest-16-01-github-output.md | 6.48 / 10.0 |
| workshop/27-evaluate-workflow-quality.md | 6.57 / 10.0 |
| workshop/23-ab-experiments.md | 6.65 / 10.0 |
| workshop/side-quest-21-01-sub-agent-syntax.md | 6.93 / 10.0 |
## Other Scores (no lesson number)

| File | Overall Score |
|---|---|
| workshop/side-quest-enterprise-setup.md | 5.71 / 10.0 |
## Score Trends (history window: 20 commits)

| File | Baseline | Latest | Delta | Direction |
|---|---|---|---|---|
| workshop/02a-setup-codespace.md | 6.09 | 6.09 | +0.00 | stable |
| workshop/04-github-actions-intro.md | 5.42 | 5.39 | -0.03 | stable |
| workshop/05-agentic-workflows-intro.md | 5.43 | 5.43 | +0.00 | stable |
| workshop/05b-agentic-workflows-security.md | 5.75 | 5.75 | +0.00 | stable |
| workshop/05c-agentic-workflows-practice.md | 6.22 | 6.22 | +0.00 | stable |
| workshop/07-your-first-workflow.md | 6.25 | 6.25 | +0.00 | stable |
| workshop/07d-confirm-model-access.md | 6.25 | 6.25 | +0.00 | stable |
| workshop/08-run-your-workflow.md | 5.67 | 5.67 | +0.00 | stable |
| workshop/08b-interpret-your-run.md | 5.99 | 5.99 | +0.00 | stable |
| workshop/09-agentic-editing.md | 5.99 | 6.01 | +0.02 | stable |
| workshop/14-next-steps.md | 5.83 | 5.83 | +0.00 | stable |
| workshop/14b-pr-reviewer-workflow.md | 5.45 | 5.45 | +0.00 | stable |
| workshop/15-conditional-logic.md | 5.53 | 5.53 | +0.00 | stable |
| workshop/16-connect-data-source.md | 5.71 | 5.71 | +0.00 | stable |
| workshop/17-add-mcp-tools.md | 5.61 | 5.61 | +0.00 | stable |
| workshop/18-share-and-reuse.md | 6.15 | 6.15 | +0.00 | stable |
| workshop/19-research-driven-training-node.md | 6.37 | 6.37 | +0.00 | stable |
| workshop/20-persistent-memory.md | 5.67 | 5.67 | +0.00 | stable |
| workshop/21-inline-sub-agents.md | 5.97 | 5.97 | +0.00 | stable |
| workshop/22-error-handling-and-resilience.md | 6.25 | 6.35 | +0.10 | stable |
| workshop/23-ab-experiments.md | 6.51 | 6.65 | +0.14 | stable |
| workshop/24-self-hosted-runners.md | 6.59 | 6.45 | -0.14 | stable |
| workshop/25-audit-and-observability.md | 6.23 | 6.15 | -0.08 | stable |
| workshop/26-manage-costs-and-budgets.md | 5.85 | 5.85 | +0.00 | stable |
| workshop/27-evaluate-workflow-quality.md | 6.57 | 6.57 | +0.00 | stable |
| workshop/28-orchestrate-workflows.md | 6.25 | 6.13 | -0.12 | stable |
| workshop/29-skills-and-domain-knowledge.md | 5.85 | 5.75 | -0.10 | stable |
| workshop/side-quest-01-01-terminal-basics.md | 6.26 | 6.26 | +0.00 | stable |
| workshop/side-quest-01-02-environment-reference.md | 4.93 | 4.93 | +0.00 | stable |
| workshop/side-quest-01-03-permission-errors.md | 6.79 | 6.53 | -0.26 | declining |
| workshop/side-quest-02-01-local-terminal.md | 6.25 | 6.25 | +0.00 | stable |
| workshop/side-quest-05-01-actions-power-user.md | 5.83 | 5.83 | +0.00 | stable |
| workshop/side-quest-05-02-aw-deep-dive.md | 6.13 | 6.13 | +0.00 | stable |
| workshop/side-quest-05-03-two-file-structure.md | 5.51 | 5.51 | +0.00 | stable |
| workshop/side-quest-06-01-install-troubleshooting.md | 6.41 | 6.41 | +0.00 | stable |
| workshop/side-quest-06-02-cca-codespace.md | 7.25 | 7.25 | +0.00 | stable |
| workshop/side-quest-06-03-copilot-token.md | 5.83 | 5.83 | +0.00 | stable |
| workshop/side-quest-06-03a-copilot-requests-permission.md | 5.75 | 5.75 | +0.00 | stable |
| workshop/side-quest-06-03b-copilot-github-token.md | 6.77 | 6.71 | -0.06 | stable |
| workshop/side-quest-06-03c-copilot-github-token-ui-only.md | 6.49 | 6.45 | -0.04 | stable |
| workshop/side-quest-06-04-install-local.md | 6.77 | 6.67 | -0.10 | stable |
| workshop/side-quest-07-01-compile-workflow.md | 6.49 | 6.45 | -0.04 | stable |
| workshop/side-quest-07d-billing-paths.md | 6.19 | 6.15 | -0.04 | stable |
| workshop/side-quest-08-01-codespaces-actions-write.md | 6.03 | 6.01 | -0.02 | stable |
| workshop/side-quest-09-01-debug-output.md | 6.15 | 6.15 | +0.00 | stable |
| workshop/side-quest-09-01a-pattern-long-plan-chain.md | 6.20 | 6.20 | +0.00 | stable |
| workshop/side-quest-09-01b-pattern-empty-results.md | 6.35 | 6.35 | +0.00 | stable |
| workshop/side-quest-09-01c-pattern-safe-output-blocked.md | 6.35 | 6.35 | +0.00 | stable |
| workshop/side-quest-09-01d-pattern-permission-denied.md | 6.37 | 6.37 | +0.00 | stable |
| workshop/side-quest-09-01e-pattern-done-no-write.md | 6.39 | 6.39 | +0.00 | stable |
| workshop/side-quest-09-01f-debugging-checklist.md | 5.96 | 5.96 | +0.00 | stable |
| workshop/side-quest-10-01-agent-brief.md | 6.09 | 6.09 | +0.00 | stable |
| workshop/side-quest-10-02-jailbreak-brief.md | 5.89 | 5.85 | -0.04 | stable |
| workshop/side-quest-10-03-skill-dispatcher.md | 5.95 | 5.95 | +0.00 | stable |
| workshop/side-quest-11-01-frontmatter-deep-dive.md | 7.03 | 6.97 | -0.06 | stable |
| workshop/side-quest-11-01b-workflow-structure.md | 5.62 | 5.59 | -0.03 | stable |
| workshop/side-quest-11-02-yaml-frontmatter.md | 7.00 | 6.88 | -0.12 | stable |
| workshop/side-quest-11-03-better-prompts.md | 6.24 | 6.24 | +0.00 | stable |
| workshop/side-quest-11-04-annotated-workflow.md | 5.72 | 5.67 | -0.05 | stable |
| workshop/side-quest-11-05-event-triggers.md | 5.87 | 5.87 | +0.00 | stable |
| workshop/side-quest-11-06-anthropic-key.md | 5.39 | 5.39 | +0.00 | stable |
| workshop/side-quest-11-07-openai-key.md | 5.91 | 5.91 | +0.00 | stable |
| workshop/side-quest-11-08-frontmatter-tools-outputs.md | 5.64 | 5.64 | +0.00 | stable |
| workshop/side-quest-11-09-agent-session-phases.md | 7.69 | 7.69 | +0.00 | stable |
| workshop/side-quest-12-01-iterate-agent-output.md | 7.91 | 7.91 | +0.00 | stable |
| workshop/side-quest-13-01-pr-labeler-pattern.md | 8.63 | 8.63 | +0.00 | stable |
| workshop/side-quest-13-01-schedule-expressions.md | 7.89 | 7.89 | +0.00 | stable |
| workshop/side-quest-13-02-pr-summary-pattern.md | 8.39 | 8.39 | +0.00 | stable |
| workshop/side-quest-13-03-pr-checklist-pattern.md | 6.45 | 6.45 | +0.00 | stable |
| workshop/side-quest-13-04-token-optimization.md | 5.67 | 5.61 | -0.06 | stable |
| workshop/side-quest-15-01-expressions-and-contexts.md | 6.05 | 6.03 | -0.02 | stable |
| workshop/side-quest-15-02-chaining-conditions.md | 6.21 | 6.21 | +0.00 | stable |
| workshop/side-quest-16-01-github-output.md | 6.48 | 6.48 | +0.00 | stable |
| workshop/side-quest-16-02-secrets-and-permissions.md | 5.79 | 5.77 | -0.02 | stable |
| workshop/side-quest-16-03-token-exfiltration.md | 5.85 | 5.79 | -0.06 | stable |
| workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md | 6.03 | 6.03 | +0.00 | stable |
| workshop/side-quest-16-05-long-lived-credentials.md | 6.01 | 5.97 | -0.04 | stable |
| workshop/side-quest-17-01-mcp-concepts.md | 5.91 | 5.89 | -0.02 | stable |
| workshop/side-quest-17-02-security-architecture.md | 5.77 | 5.77 | +0.00 | stable |
| workshop/side-quest-17-03-prompt-injection.md | 5.81 | 5.81 | +0.00 | stable |
| workshop/side-quest-17-04-permission-escalation.md | 5.93 | 5.93 | +0.00 | stable |
| workshop/side-quest-17-05-supply-chain-mcp.md | 6.01 | 6.01 | +0.00 | stable |
| workshop/side-quest-17-06-output-injection.md | 5.97 | 5.97 | +0.00 | stable |
| workshop/side-quest-17-07-repo-poisoning.md | 4.97 | 4.97 | +0.00 | stable |
| workshop/side-quest-20-01-memory-patterns.md | 5.57 | 5.57 | +0.00 | stable |
| workshop/side-quest-21-01-sub-agent-syntax.md | 6.93 | 6.93 | +0.00 | stable |
| workshop/side-quest-22-01-failure-modes.md | 5.57 | 5.57 | +0.00 | stable |
| workshop/side-quest-23-01-ab-roundrobin.md | 5.83 | 5.83 | +0.00 | stable |
| workshop/side-quest-24-01-runner-infrastructure.md | 6.13 | 6.13 | +0.00 | stable |
| workshop/side-quest-25-01-audit-reference.md | 5.91 | 5.93 | +0.02 | stable |
| workshop/side-quest-26-01-forecast-costs.md | 6.24 | 6.20 | -0.04 | stable |
| workshop/side-quest-29-01-skill-injection-strategies.md | 6.11 | 6.09 | -0.02 | stable |
| workshop/side-quest-enterprise-setup.md | 5.75 | 5.71 | -0.04 | stable |
## Overall Trend

- Baseline mean score: `6.17`
- Latest mean score: `6.14`
- Delta: `-0.03`
- Direction: `stable`

## Bloom's Taxonomy Distribution

| Level | File Count |
|---|---|
| Remember | 8 |
| Understand | 15 |
| Apply | 50 |
| Analyze | 7 |
| Evaluate | 5 |
| Create | 8 |

Pyramid gap check: `has_min_understand_steps = true`, `has_min_analyze_steps = true` — no corpus-level Bloom pyramid gap detected. The distribution is Apply-heavy (54% of files) but clears the minimum thresholds for Understand and Analyze/above, so no `gap_proposals` were generated.

## ⚠️ Rubric Instrumentation Issue (affects all scores above)

While validating findings against source files, we found that `CHECKPOINT_RE` and the prerequisite-section regex in `curriculum_assessment.py` only match literal emoji characters (`✅`, `📋`), but 88 of 93 workshop files use GitHub's `:white_check_mark:` / `:clipboard:` shortcode syntax per the repo's own style conventions. As a result:
- `has_checkpoint` is `false` for 88/93 files even though every one of those files has a real `## :white_check_mark: Checkpoint` section with checklist items — `checkpoint_quality` (weight 2.0) is silently zeroed out corpus-wide.
- `has_prereq_section` is `false` for all 93/93 files even though 86 files have a `## :clipboard: Before You Start` section — `scaffolding` (weight 1.5) is capped at 5/10 for every file that actually has scaffolding.

This means the **`overall_score` values in this scorecard understate true quality** for nearly the whole corpus and should be treated as a lower bound until the regex is fixed. A recommended fix is filed as a separate issue below.

> Generated by [🔬 Curriculum Quality Evaluator](https://github.com/githubnext/gh-aw-workshop/actions/runs/35098365907) · copilot · auto · 88.8 AIC · ⌖ 13.7 AIC · ⊞ 9.5K · [◷](https://github.com/search?q=repo%3Agithubnext%2Fgh-aw-workshop+is%3Aissue+%22gh-aw-workflow-call-id%3A+githubnext%2Fgh-aw-workshop%2Fcurriculum-evaluator%22&type=issues)
> - [x] expires on Sep 17, 2026, 1:01 PM UTC

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the score tables and 20-commit trend history in this issue, then inspect the listed workshop/*.md files, such as workshop/02a-setup-codespace.md and workshop/side-quest-01-03-permission-errors.md. The issue does not identify a target file, requested change, or completion criteria, so the desired output must be clarified before implementation.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.