sphinx-doc / sphinx-doc/sphinx
Regex change in #8225 causes failure to load certain objects with full signatures containing integers in object name
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
As currently implemented to avoid the potential regex DOS reported in #8175, the regex in InventoryFile.load_v2() will incorrectly parse certain object data lines with space-delimited integers in the name portion of the line.
One particular objects.inv file that exhibits this behavior is an old OpenCV v2.4 inventory, from which one offending line is:
virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0 ocv:function 1 modules/gpu/doc/video.html#$ -
The 0 of this line is mis-interpreted as the priority value; see below.
To Reproduce
Steps to reproduce the behavior:
>>> import re
>>> pattern = re.compile(r'(?x)(.+?)\s+(\S+)\s+(-?\d+)\s+?(\S*)\s+(.*)') # from 3.3.1
>>> pattern.match("virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0 ocv:function 1 modules/gpu/doc/video.html#$ -").groups()
('virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize)', '=', '0', 'ocv:function', '1 modules/gpu/doc/video.html#$ -')
domain:role is found (and rejected, having no colon) as =, priority as 0, uri as ocv:function, etc.
Expected behavior
The data line is valid and should be parsed correctly. E.g., the regex from Sphinx v3.0.4 works fine:
>>> import re
>>> pattern = re.compile(r'(?x)(.+?)\s+(\S*:\S*)\s+(-?\d+)\s+?(\S*)\s+(.*)') # from 3.0.4
>>> pattern.match("virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0 ocv:function 1 modules/gpu/doc/video.html#$ -").groups()
('virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0', 'ocv:function', '1', 'modules/gpu/doc/video.html#$', '-')
Possible fixes
Changing the lazy repetition for name to a regular repetition (removing the ?) seems to work ok, at least for this data line:
>>> import re
>>> pattern = re.compile(r'(?x)(.+)\s+(\S+)\s+(-?\d+)\s+?(\S*)\s+(.*)') # Modified from 3.3.1
>>> pattern.match("virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0 ocv:function 1 modules/gpu/doc/video.html#$ -").groups()
('virtual uchar* gpu::VideoWriter_GPU::EncoderCallBack::acquireBitStream(int* bufferSize) = 0', 'ocv:function', '1', 'modules/gpu/doc/video.html#$', '-')
But, this might not be a general fix, and it may be necessary to implement one of the regexes suggested by @yetingle in #8175.
This is a fairly esoteric bug; if it's desired to fix it, I can do the legwork of validating candidate regexes for a sampling of objects.inv files, in addition to the checks provided by Sphinx's test suite -- I check a bunch of them as part of my test suite for sphobjinv.
Your project
N/A, internal bug.
Screenshots
N/A
Environment info
- OS: Windows
- Python version: 3.9.1
- Sphinx version: 3.3.1 and 3.4.3
- Sphinx extensions: N/A, but relevant to intersphinx
- Extra tools: N/A
Additional context
Add any other context about the problem here.
- Bug believed introduced by #8225, in response to #8175
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sphinx/util/inventory.py at InventoryFile.load_v2() and reproduce the parsing failure with the linked OpenCV objects.inv line. Compare candidate regexes against the Sphinx test suite and the referenced sphobjinv inventory samples. Done means valid lines with integers in object names parse into the correct name, domain:role, priority, URI, and display name without reintroducing the regex DOS concern.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100