python / python/cpython

The elements in `Grammar/python.gram` stripped by `peg_lexer.py` still shows in generated PO files

Open
#142,714 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

docs topic-parser
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

My original question has answered by Stan. I leave it here as a record:

Click to expand

I'm not sure if it's done deliberately, but it seems odd to me and I can't find out why it happens so far.


The reference/grammar.rst includes Grammar/python.gram

https://github.com/python/cpython/blob/78a50ee10ef73296eba98899d496bff3915ddc41/Doc/reference/grammar.rst#L18-L19

and Grammar/python.gram has a section for invalid rules at the bottom.

https://github.com/python/cpython/blob/78a50ee10ef73296eba98899d496bff3915ddc41/Grammar/python.gram#L1193-L1202

However, the content after START OF INVALID RULES is missing on the page

snapshot Image

I checked the previous version and found out the doc has this issue since 3.11 (after PEG parser is introduced).


Other info:
I translated this page into Traditional Chinese (zh-tw) recently and somehow it shows correctly on the translated page

snapshot Image

So here's the revised issue description:

Some elements in Grammar/python.gram are stripped by peg_lexer.py, and hence they're hidden on the reference/grammar docs. However, the full content of Grammar/python.gram is still remained in the generated PO files (e.g. the one in zh-tw).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Doc/tools/extensions/lexers/peg_lexer.py and compare its handling of Grammar/python.gram with the inclusion in Doc/reference/grammar.rst. Inspect the generated reference/grammar.po example to understand where the stripped content remains. Done should define and consistently apply the expected treatment in both rendered documentation and generated PO files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.