hoangsonww / hoangsonww/Forge-Agentic-Coding-CLI

Feature: Validation Recipe Registry with Project-Specific Quality Profiles

Open
#44 0 comments 0 reactions 1 assignee Claimed by @hoangsonww View on GitHub
bug documentation enhancement good first issue help wanted question
Dominant language
TypeScript
Stars
23
Forks
9
PR merge metrics
No merged PRs in 30d

Description

## Summary

Add a Validation Recipe Registry so Forge can discover, preview, select, and run project-specific quality profiles instead of relying only on generic package-script detection during validation gates.

## Problem / Opportunity

Forge already has a validation gate and documented canonical verification chain, but real projects often need more nuanced quality profiles than a single inferred command set. Examples include fast local checks, full CI parity, docs-only checks, Docker checks, release checks, and language-specific recipes. Without a registry, users have limited visibility into why Forge chose a validation command, how expensive it may be, or how to pin a project-specific profile that agents can reuse consistently.

This is distinct from release readiness dashboards or spec acceptance gates: the goal is a reusable local validation abstraction that the agentic loop, CLI, dashboard, and documentation can all share.

## Proposed Feature

Introduce validation recipes as named, typed quality profiles that Forge can detect, explain, and execute:

- Add a registry of built-in recipes for common project types such as Node/TypeScript, Python, Rust, Go, Docker, docs-only, and Forge's own canonical chain.
- Add project config support for custom recipes with command lists, expected duration, mutation policy, environment requirements, and failure parsing hints.
- Add CLI commands to list, explain, dry-run, and run recipes.
- Teach the validation gate to select the best recipe based on changed files, mode policy, project config, and explicit user override.
- Persist validation runs as structured events so dashboard/task replay can show which recipe ran and why.
- Document how contributors add a recipe without bypassing permission checks or introducing network-dependent tests.

## Scope

- `src/core/validation.ts` selection logic and result metadata.
- Config schema additions for project-level validation recipes.
- CLI commands under a `validation` or existing doctor/verify surface.
- Dashboard/API support for viewing available recipes and recent validation runs.
- Built-in recipes with conservative defaults and no surprise network calls.
- Tests covering recipe discovery, precedence, dry-run output, changed-file matching, and failure reporting.
- Documentation in README, CLI reference, setup docs, and contributor guidance.

## Acceptance Criteria

- [ ] `forge validation list` shows built-in and project-defined recipes with descriptions and estimated cost/time.
- [ ] `forge validation explain ` shows commands, match rules, required tools, mutation/network posture, and why it is applicable.
- [ ] `forge validation run --dry-run` prints the exact commands without executing them.
- [ ] The agentic validation gate records the selected recipe id, selection reason, commands, exit status, duration, and parsed failures.
- [ ] Project config can pin a default recipe and define custom recipes without changing source code.
- [ ] Changed-file-aware selection can choose lighter docs-only or test-only recipes when safe, while still allowing a full profile override.
- [ ] Every command execution still goes through the existing permission/risk classification path.
- [ ] Unit tests cover registry behavior, config validation, changed-file matching, and failure serialization.

## Non-Goals

- Replacing CI providers or GitHub Actions.
- Guaranteeing perfect command selection for every language ecosystem in the first release.
- Adding hosted telemetry, remote build execution, or network-dependent recipe checks.
- Changing Forge's canonical verification requirement for this repository.

## Dependencies / Risks

- Recipe selection must stay deterministic enough for replay and audit workflows.
- Built-in recipes should avoid hiding failures by being too clever with changed-file filtering.
- Command metadata must integrate with existing sandbox and permission classification rather than introducing a parallel execution path.
- Failure parsing should be helpful but must preserve raw output links or excerpts for debugging.

## Open Questions

- Should recipes be stored only in Forge config, or should Forge also read common files such as `package.json`, `pyproject.toml`, and `Makefile` as recipe sources?
- Should the first dashboard version only display recipe results, or also allow selecting and running recipes interactively?
- What minimum metadata should a custom recipe require before Forge treats it as safe for automated validation retries?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.