AbsaOSS / AbsaOSS/living-doc

`collector-gh` `doc-issues`: local-directory source + canonical issue-body parser

Aperta
#19 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
0
Fork
0
Merge medio
22h 13m
PR unite (30g)
11

Descrizione

## Background

`.github/workflows/real-collector-snapshot.yml` snapshot-tests the **real**
`collector-gh` over `docs/examples/gherkin/` for the `doc-source` and `ui-tests`
modes, plus `toolkit coverage-matrix`. Issue #19 asks to extend the same job to the
third mode — `doc-issues` → `toolkit normalize-issues` → `generator-ready.json` —
using the committed corpus `docs/examples/gh-issues/*.md` as the source.

That cannot be done today. Investigation on 2026-09-10 (against `collector-gh`
master tip `b6c935d`, the currently pinned ref) found two gaps:

1. **No local input for `doc-issues`.** The mode takes only
`doc-issues-repositories` (`organization-name` + `repository-name`) and mines via
the GitHub API. `doc-source` / `ui-tests` already accept local paths
(`us-paths` / `func-paths` / `paths`), which is what makes their snapshot job
reproducible and offline. `doc-issues` has no equivalent.
2. **Parser does not match the canonical issue-body layout.**
`doc_issues/body_parser.py` extracts `description` / `business_value` /
`preconditions` and an `## Acceptance Criteria` **table**. The canonical layout
pinned in `docs/examples/README.md` (and used by the corpus) is instead:
- entity **type** from the issue **label**
(`DocumentedUserStory` / `DocumentedFeature` / `DocumentedFunctionality`),
- entity **ID** from the **title** prefix (`US-001 · Customer Login`),
- `##` section headings = Title-Case of the toolkit synonym-table key
(`## User Stories`, `## Functionalities`, `## Parent Feature`,
`## Not In Scope`, `## Business Value`, …),
- AC blocks as `###` sub-headings with the glossary grammar
`AC: (v - )`.

Until both land, `docs/examples/gh-issues/*.md` is a **specification of what
`doc-issues` should mine**, not a source it can mine.

## Decision (2026-09-10)

Issue #19 is **deferred**. We will not stand up a partial or mocked `doc-issues`
snapshot leg in `living-doc`. Instead:

- Land this prerequisite in `collector-gh` first.
- Then implement #19 in one pass as a full **ecosystem integration test**: all three
collector modes over the shared corpus, `normalize-issues` → `generator-ready.json`
snapshot, and (optionally) a `generator-pdf --document-type technical-project`
render check.

## Scope — `collector-gh`

- [ ] `doc-issues` accepts a local source: a directory of issue-body `*.md` files
(one file per issue), mirroring how `doc-source` takes local paths. Each file
supplies the body; label and title come from front-matter or an agreed
convention (see the HTML-comment header already in the corpus files).
- [ ] Body parser implements the canonical layout from
`docs/examples/README.md` § "GitHub issue-body layout (canonical)":
label→type, title-prefix→ID, `##` synonym headings, `### AC: (v - )`
blocks, feature-level `Preconditions` / `Not In Scope` inherited by ACs.
- [ ] `doc-issues.json` output validates against
`doc_issues/schema/doc-issues-v1.0.0-schema.json` (regenerate the schema if
models move).
- [ ] Unit coverage for the new parser over the three corpus files.

## Scope — `living-doc` (follow-up, was #19)

- [ ] Bump the pinned `COLLECTOR_GH_REF` to the ref that ships the above.
- [ ] Extend `tools/regen-collector-snapshots.sh` to also run `doc-issues` over
`docs/examples/gh-issues/` and `toolkit normalize-issues` over its output.
- [ ] Snapshot `generator-ready.json` (normalized via `tools/normalize_snapshot.py`)
against `docs/examples/_expected/generator-ready.json`.
- [ ] Add `GENERATOR_PDF_REF` to `tools/collector-snapshot-pins.env`; optionally
chain `generator-pdf --document-type technical-project` and snapshot its
`meta` / warnings (never byte-compare the PDF).
- [ ] Document the regen command in `CONTRIBUTING.md`.

## Acceptance criteria

- [ ] `collector-gh` `doc-issues` mines `docs/examples/gh-issues/*.md` offline and
emits schema-valid `doc-issues.json`.
- [ ] The parser round-trips the canonical layout (type, ID, sections, AC grammar).
- [ ] #19 can then be implemented with no mocking: real collector → real toolkit →
committed expected snapshot, green on `master`.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.