eellak / eellak/glossAPI

Phase A reflow may shift HTML-comment line numbers used by math-enhance replacement step

Open
#97 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
141
Forks
41
PR merge metrics
No merged PRs in 30d

Description

## Context

`corpus.extract --math-enhance` emits a JSON describing positions of math-equation placeholders in the extracted markdown. Each placeholder lives in the MD as an HTML comment (e.g. ``, ``). A later step uses that JSON to:

1. Locate the placeholder in the MD,
2. OCR the corresponding equation from the PDF using a math model,
3. Replace the placeholder comment with the OCR'd LaTeX/MathML.

`corpus.clean` is being updated to optionally run Phase A reformatting (`PhaseAMode::ParserSurgicalVerified`, the parser-backed surgical rewriter from `md_format_surgical`) BEFORE the destructive cleaning passes. The full design is in `rust/glossapi_rs_cleaner/docs/PHASE_A_PARSER_BACKED_*.md`.

## Concern

Phase A reflow can shift the LINE NUMBER of an HTML comment that's inline within a paragraph. Concretely: if a 5-line soft-wrapped paragraph contains an inline `` on line 4, paragraph reflow joins those 5 lines into 1 — the comment is now on a different line in the output, even though its byte content is byte-exact preserved.

Verified properties of Phase A on HTML comments:

- Comment **bytes are preserved byte-exact** (rewrite only touches `\n` at paragraph soft-break boundaries; never modifies comment payload).
- Comment **block structure is preserved** (`HtmlBlock` nodes pass through verbatim; inline comments inside `Paragraph` content are kept inline).
- Comment **line number is NOT preserved** when an enclosing paragraph reflows.

## Action items (low priority, future work)

1. Audit how the math-enhance replacement step locates placeholders:
- If it greps by **comment text / stable ID** (e.g. ``) → no impact, safe to enable Phase A before it.
- If it locates by **(line, column) from the extract-time JSON** → would break under Phase A reflow. Two options:
a. Run math-enhance replacement BEFORE Phase A (current default ordering — keep this).
b. Switch math-enhance to use stable text-anchor IDs and re-emit positions after Phase A.

2. Document the chosen ordering / contract in the cleaner config + the math-enhance pipeline docs.

3. Add a regression test: a doc with inline math-equation comments → run through Phase A → assert each comment text still appears in output (byte-exact, position may differ).

## References

- Phase A implementation: \`rust/glossapi_rs_cleaner/src/md_format_surgical.rs\`
- Verified safety on 240 hardest-altered docs: 0 comment-loss bugs.
- Full architecture: \`rust/glossapi_rs_cleaner/docs/PHASE_A_PARSER_BACKED_INDEX.md\`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.