pprose (practical-prose) use case: document-model needs and a few gaps after the 0.1.0 migration
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Context: the practical-prose / pprose use case
[practical-prose](https://github.com/jlevy/practical-prose) ships **pprose**, an
evaluation toolkit for practical documents. It computes a structural + linguistic
metrics report over Markdown (`pprose metrics`), and is building a two-phase AI-prose
linter that flags constructions by exact source span and verifies each candidate with a
focused model call.
pprose just migrated from `chopdiff==0.3.1` to `flexdoc==0.1.0` and dropped chopdiff
entirely (it only ever used the document model, not the diff/windowed-transform layer).
The migration was a clean drop-in: `chopdiff.docs.TextDoc` -> `flexdoc.FlexDoc`,
identical word/sentence/paragraph/line counts, and the fixture-locked metrics test passed
byte-for-byte. Thank you for the clean extraction.
This issue records the full set of document-model needs pprose has, so flexdoc can be a
complete foundation for "comprehensive review and control of practical documents." Most
of it already works in 0.1.0; the genuine gaps are a short, prioritized list at the end.
## What already works well in 0.1.0 (no action needed)
Verified empirically against the published 0.1.0 wheel:
- **Prose-only sizing** via `FlexDoc.filtered(include={...}).size(TextUnit.{words,sentences,paragraphs})`.
- **Frontmatter isolation** (`FlexDoc.frontmatter`; excluded from `paragraphs` and all counts).
- **Typed block metadata**: `Block.code_info` (language, line_count), `Block.table_info`
(rows, cols, cells, alignments), `Block.list_info` (ordered, start, max_depth, item_count).
- **Sections + rollups**: `sections()`, `Section.size(unit, subtree=)`, `section_size_tree()`, `toc()`.
- **Inline node collection** including **footnote references**: `collect(kinds={NodeKind.footnote_ref | link | image | code_span}, recursive=True)` returns typed nodes with exact spans and useful attrs (`url`, `text`, `label`, `sentence`, `section`).
- **Exact offsets + inversion**: `Offsets` on every paragraph/sentence/inline node; `paragraph_at_offset`, `sentence_at_offset`.
- **`SpanRef`** quote-canonical anchoring for span-stable annotations.
Notably, this **closes all five** of the old chopdiff feature requests
(jlevy/chopdiff#18-#22), including `NodeKind.footnote_ref`, which is implemented in 0.1.0.
## Requests (prioritized)
### 1. Typed link *form* discriminator (highest value)
pprose reports links broken down by Markdown form: **inline** `[t](u)`, **autolink**
``, **bare URL** `https://...`, **reference-use** `[t][id]`, **reference-definition**
`[id]: url`, and **image** ``. In 0.1.0 every one of these (except image) collapses
to `NodeKind.link` with attrs `{url, text, sentence}` and no way to tell them apart, and
**reference definitions are not surfaced at all** (resolved into the use's `url`, no node
or count). A bare URL is also returned as a `link` (GFM autolinking), which conflicts with
pprose's "bare URL = not in link syntax" metric.
Request: a typed `form` on link nodes / `Link`, e.g.
`Literal["inline","autolink","bare_url","reference","image"]`, so consumers can count by
form without heuristics (today the only signal is `text == url`, which cannot separate
autolink from bare URL, nor reference-use from inline).
### 2. Surface link reference definitions
Expose `[id]: url` definition lines, either as a node kind (e.g. `link_ref_def`) or a
count/list on `FlexDoc`. pprose currently regexes these for its `links_reference_definitions`
metric; flexdoc resolves them away with no typed handle.
### 3. `block_at_offset` (offset -> block inversion)
The linter maps a match offset back to its enclosing block to decide skip rules
(code/table/frontmatter) and build context windows. `paragraph_at_offset` and
`sentence_at_offset` exist; a `block_at_offset` (or a documented "use
`collect(overlaps=(o, o+1), layer={Layer.markdown})`" recipe) would complete the set.
Low priority since it is derivable.
### 4. Fuzzy / edit-distance `SpanRef` re-anchoring (forward-looking)
Already on the roadmap (spec §14). The two-phase linter applies corrections and re-runs;
corrections must re-anchor after the text around them shifts. A +1 with this concrete use
case.
## Ergonomics note (not a bug)
Inline collection requires `recursive=True`. `collect(kinds={NodeKind.link}, inline=True)`
alone returns `[]` because, with no tree/interval relation, candidates default to root
(block) nodes and inline nodes are nested. This is easy to trip over; a one-line note in
`collect()`'s docstring or `usage.md` (or having explicit inline kinds imply `recursive`)
would save the next consumer the same detour.
## Out of scope for flexdoc (recorded for completeness)
pprose's lint-pattern metrics (banned register, spaced-em-dash discipline,
replacement-history / pedantic-marker phrases, ALL-CAPS bracket tags, generic headings)
are editorial heuristics, not document structure. pprose will keep running these as
regexes over the prose-only `filtered(...).reassemble()` text; flexdoc need not type them.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.