Configurable size-counting scopes (prose-only, with/without tables/inline-code) without re-parsing
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Context
[pprose](https://github.com/jlevy/practical-prose) computes document metrics and just migrated its sizing onto flexdoc 0.2.0 (thanks!). In doing so we hit a counting-scope gap: there's no clean, configurable way to get **size counts (words / sentences / paragraphs / lines) for a specific prose scope**, and the scope you want genuinely differs by use case. Filing as a feature request; not blocking (we have a working approach), but it'd let consumers stop composing ad-hoc projections.
## The problem
We want size counts over the *same* prose scope our lint runs over (so `em_dash_density = em_dashes / words` has a consistent numerator and denominator). The natural prose projection is `prose_text()`, but the available options each fall short:
- **`filtered(include={paragraph, list, ...}).size()`** sizes a real sub-document (good, single parse), but the scope is fixed by which `BlockType`s you include: it excludes heading and table text, and it *keeps* inline-code spans inside paragraphs (so `` `code` `` tokens count as words). That's the wrong scope for our case and not adjustable along the axes we care about.
- **`prose_text(include_tables=…)`** is exactly the scope we want (drops inline code, unwraps links/images, excludes frontmatter + code blocks, tables optional) — but it returns a **string**, so to get sizes we have to **re-parse** it: `FlexDoc.from_text(doc.prose_text(...)).size(...)`. That's a second parse of derived text, and a bit circular.
So today the choices are "a sub-document with the wrong scope" or "the right scope but only as text, requiring a re-parse."
## What would help (not prescriptive)
A way to get **sizes for a prose projection without re-parsing**, with the scope **configurable per use case**. Concretely, the axes that matter to us:
- **Tables in or out.** Word counts *with* and *without* table content are both legitimately useful (table-cell text is content for some metrics, noise for prose-density ratios). We'd like to ask for either.
- **Inline code in or out.** For prose word counts, inline-code tokens are usually noise; for other uses they're content.
- **Headings in or out**, similarly.
Whatever shape fits flexdoc best — e.g. a `size(unit, scope=…)` with named scopes, options on `filtered()`/`prose_text()` that also expose `.size()`, or a small `ProseScope` config — so a consumer can say "words, prose-only, without tables" and get a consistent answer without re-parsing or hand-assembling `BlockType` sets.
## What "done" looks like for us
- Get words / sentences / paragraphs / lines for a named/parameterized prose scope in one call, no re-parse.
- Select table inclusion (and ideally inline-code / heading inclusion) per call.
- The scope used for sizing is the same one `prose_text()` exposes, so text-based lint and numeric sizes stay consistent.
## Current pprose workaround
We size over `FlexDoc.from_text(doc.prose_text(include_tables=True)).size(...)` and run lint over the same `prose_text(...)` string, which keeps the two consistent. It works; the re-parse and the fixed scope are the rough edges this request would smooth.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.