eclipse-score / eclipse-score/score
Standardize Evidence Artifacts and Schema
- Dominant language
- Starlark
- Stars
- 109
- Forks
- 105
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 21
Description
## Goal
Standardize the artifact set emitted by all onboarded repositories.
## Why
Developers and governance tooling need consistent machine-readable evidence across repos.
## Required Artifact Contract
**Base fields (all repos)**
- `execution_lane` (`oss_default` | `accelerated`)
- `policy_decision` (allow | deny, with machine-readable reason)
- `test_summary`
- `quality_or_traceability_metrics`
- `dependency_manifest_summary`
- `license_policy_result`
- `reproducibility_metadata` (rerun id, tool versions, environment hash)
**Extended fields for docs-as-code / assurance domain**
- `impacted_elements` — list of need IDs or argument element IDs directly impacted by the change
- `impact_class` per element — one of: `direct_recheck` | `indirect_propagation` | `revision_required`
- `consistency_rule_ids` — which rules triggered the impact classification
- `affected_artifact_type` — `requirement` | `guideline` | `test_link` | `process_requirement` | `evidence`
- `change_scenario_id` — reference to the specific change scenario class that was matched
- `responsible_role` — who is accountable for the outcome (default: `pr_creator`)
- `escalation_role` — who to escalate failures to (default: `harness_maintainer`)
- `waiver_authority` — who can approve waivers (default: `release_approver`)
**Provenance fields** (all repos):
- `execution_timestamp` — ISO 8601 timestamp
- `python_version` — interpreter version
- `environment_hash` — stable hash of execution environment
- `gate_script_version` — validator version
These extended fields enable the harness outer loop to grep structured traces and identify recurring failure patterns without reading raw sphinx-build output.
## Queryability Rules
- Artifacts must be machine-readable (`json`, `jsonl`, or schema-validated yaml)
- Filenames must be stable and consistent across runs so simple tools such as `grep` and `cat` work reliably
- Raw stdout/stderr may be retained as optional debug artifacts, but never as the primary evidence contract
- A small summary index (`evolution_summary.jsonl` or equivalent) must exist so agents start from the index before drilling into traces
- The docs-as-code pilot should standardize at least `gate_output.json`, `impacted_elements.json`, and `score.json` per task
## Tasks
- Define artifact manifest and schema versions.
- Reuse/align with existing docs-as-code schema patterns.
- Validate schema compatibility in CI.
- Document retention and discoverability.
- Define compatibility policy for schema evolution.
- Define canonical filenames and directory layout for per-run and per-task trace artifacts.
## Docs-as-code pilot tasks
- Extend current metrics artifacts with required provenance fields.
- Validate artifact schemas in CI before policy decision.
- Ensure accelerated-lane artifacts remain verifiable by OSS validators.
- Add `evolution_summary.jsonl` or equivalent summary index for selective navigation of prior runs.
- Document the canonical per-task trace layout under `runs///traces//`.
## Done When
- Pilot repos emit the same artifact contract.
- CI rejects malformed or incomplete artifact payloads.
- Schema version policy is documented and enforced.
- Agents can locate the relevant trace files for one failure class without reading raw command output dumps.
Parent: eclipse-score/score#2852
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.