redhat-developer / redhat-developer/rhdh-plugins
Extend the scorecard github module with a new AI adoption metric provider
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 48
- Forks
- 120
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 337
Description
Scorecard is a backstage plugin and supports multiple metric, grouped as modules in folder workspaces/scorecard/plugins.
Extend the existing GitHub plugin/module under plugins/github with a new GithubAiAdoptionMetricProvider.
-
getProviderDatasourceId: "github" -
getProviderId: "github.aiAdoption" -
getMetrics: 3 metrics, for the AI adoption rate for the last 7 days, 30 days and 90 days. ID should be calculated from this.getProviderId() + "Rate[7d]" (etc.). The threshold for all 3 metrics should be defined in the provider with:export const AI_ADOPTION_RATE_TIME_RANGES = ["7d", "30d", "90d"]; export const AI_ADOPTION_RATE_THRESHOLD: ThresholdConfig = { rules: [ { key: 'success', expression: '>=0.2' }, { key: 'warning', expression: '>=0.1' }, { key: 'error', expression: '>=0' }, ], }; -
catalogFiltersimilar to the other GitHub providers:getCatalogFilter(): Record<string, string | symbol | (string | symbol)[]> { return { 'metadata.annotations.github.com/project-slug': CATALOG_FILTER_EXISTS, }; } -
calculateMetricsshould request the commit history via GraphQL from GitHub for the biggest time range that is defined (currently 90 days).Then it should take a look into all received commit messages and check if they was AI assisted or not. It could probably ignore merge commits.
It should check commit message lines that starts
Assisted-by:orCo-Authored-By:(orCo-authored-by:). When the value starts with a known AI tool, like Claude, Cursor it should count this message as AI assisted. Add extra conditions that identifies clearly AI assistents.Log at the end how many commits was analyied, how many commits was ignored, marked as ai assisted commits and NOT marked as ai assisted commits.
Finally return the Map with the metric Ids and the ratio (0-1) per time range.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing GitHub providers under workspaces/scorecard/plugins/github and trace how they fetch commit history through GraphQL and expose metric maps. Done means a GithubAiAdoptionMetricProvider defines the requested IDs, thresholds, catalog filter and 7d/30d/90d ratios, while handling qualifying commit messages and logging the requested counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, graphql, typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 56/100