eclipse-score / eclipse-score/score

Build Shared CI Workflow for Agent Runs

Open
#2,854 0 comments 0 reactions 0 assignees View on GitHub
community:ai
Dominant language
Starlark
Stars
109
Forks
105
Avg merge
1d 9h
Merged PRs (30d)
21

Description

## Goal
Provide reusable workflow/action(s) for agent execution and validation.

## Why
Developers need one standard pipeline shape across repos with clear required checks and artifact outputs.

## Tasks
- Add reusable workflow for the mandatory Lane A path.
- Integrate stages: build -> test -> policy -> artifact publish.
- Normalize failure reasons and status names across repos.
- Document adoption steps for maintainers.
- Add optional Lane B integration hooks that never affect merge eligibility.
- Add a lightweight candidate validation step before any expensive benchmark or full task-set run.

## Required Pipeline Outputs
- Policy decision artifact
- Test summary artifact
- Traceability/quality metrics artifact
- Run metadata (commit, lane, tool versions)
- Validation status artifact for the cheap pre-benchmark candidate check

## Docs-as-code pilot tasks
- Implement workflow job sequence using existing traceability scripts.
- Upload metrics and gate outputs as artifacts.
- Expose check names intended for branch protection.
- Add a fast harness validation step: import candidate, instantiate harness, run one tiny task spec, verify expected trace filenames.
- Reuse the pilot's small summary-first trace layout so failed tasks can be inspected without replaying a whole run.

## Done When
- At least 2 pilot repos run the shared workflow end-to-end.
- Required check names are stable and documented.
- Lane B can be disabled without breaking merge-critical checks.
- Malformed candidates fail in the cheap validation stage before they consume benchmark time.
- The pilot workflow emits enough structured trace metadata for agents and maintainers to inspect failures from an index before drilling into per-task artifacts.

Parent: eclipse-score/score#2852

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.