openai / openai/codex

readme_toc.py misparses tilde and longer Markdown fences

Open Beginner friendly
#41,077 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug documentation
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

Summary

The README TOC generator only recognizes backtick fences and treats every line beginning with three backticks as a close. It therefore includes headings from tilde fences and from content inside a longer backtick fence.

Reproduction

Run the README TOC generator on:

## Before
~~~markdown
## Not a heading

After


Also try a four-backtick fence containing a literal triple-backtick line.

Expected: headings inside either fenced block are excluded, and the longer fence remains open until a closing fence with the same marker and at least the same length.

Actual: headings inside tilde fences are treated as headings, and a triple-backtick content line can prematurely close a longer backtick fence.

## Root cause

The parser used a boolean toggled by `line.strip().startswith("```")`; it did not track fence marker type or length.

## PR-ready reference

I prepared a tested reference branch containing a minimal parser fix and regression tests:

- Commit: https://github.com/Hughhhhcoder/codex/commit/57516831cf353aab3a73c4e99136ab987a00d1db
- Branch: `codex/codex-readme-fence`

The patch tracks the fence character and opening length, ignores backtick openers whose info string contains a backtick, and accepts only valid matching closers.

## Validation

`python -m pytest -c /tmp/codex-empty.ini --noconftest -q scripts/test_readme_toc.py` — 3 passed.

Because the current contribution policy does not accept external code contributions or pull requests, I am submitting this analysis and reference commit here instead of opening a PR. Please feel free to implement or adapt the fix.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with scripts/test_readme_toc.py and run the documented pytest command, then follow it to the README TOC generator. Confirm the parser handles tilde fences and longer backtick fences as described, with headings inside excluded and the regression tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.