readme_toc.py misparses tilde and longer Markdown fences
Open
Beginner friendly
Nobody has claimed this yet.
bug
documentation
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.5k
- PR merge metrics
- PR metrics pending
Description
Summary
The README TOC generator only recognizes backtick fences and treats every line beginning with three backticks as a close. It therefore includes headings from tilde fences and from content inside a longer backtick fence.
Reproduction
Run the README TOC generator on:
## Before
~~~markdown
## Not a heading
After
Also try a four-backtick fence containing a literal triple-backtick line.
Expected: headings inside either fenced block are excluded, and the longer fence remains open until a closing fence with the same marker and at least the same length.
Actual: headings inside tilde fences are treated as headings, and a triple-backtick content line can prematurely close a longer backtick fence.
## Root cause
The parser used a boolean toggled by `line.strip().startswith("```")`; it did not track fence marker type or length.
## PR-ready reference
I prepared a tested reference branch containing a minimal parser fix and regression tests:
- Commit: https://github.com/Hughhhhcoder/codex/commit/57516831cf353aab3a73c4e99136ab987a00d1db
- Branch: `codex/codex-readme-fence`
The patch tracks the fence character and opening length, ignores backtick openers whose info string contains a backtick, and accepts only valid matching closers.
## Validation
`python -m pytest -c /tmp/codex-empty.ini --noconftest -q scripts/test_readme_toc.py` — 3 passed.
Because the current contribution policy does not accept external code contributions or pull requests, I am submitting this analysis and reference commit here instead of opening a PR. Please feel free to implement or adapt the fix.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/test_readme_toc.py and run the documented pytest command, then follow it to the README TOC generator. Confirm the parser handles tilde fences and longer backtick fences as described, with headings inside excluded and the regression tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100