davidhalter / davidhalter/parso

Uncaught exception in get_normalizer_issues

Open
#219 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
679
Forks
120
PR merge metrics
No merged PRs in 30d

Description

Hello,

We have been trying to fuzz `parso` to help catch any uncaught exceptions that may occur. We found that the following program generates an uncaught exception:

```python
import sys
import atheris
import parso

def TestOneInput(data):
fdp = atheris.FuzzedDataProvider(data)
try:
grammar = parso.load_grammar()
module = parso.parse(fdp.ConsumeUnicodeNoSurrogates(sys.maxsize))
if module != None:
grammar._get_normalizer_issues(module)
except RecursionError:
# Not interesting
pass

data = (b"\xff\x5c\x0a\x23")
TestOneInput(data)
```

The Atheris module used is https://pypi.org/project/atheris/ `FuzzedDataProvider` is used to convert the raw bytes in `data` into appropriate types.

If we write out the value generated by `fdp.ConsumeUnicodeNoSurrogates` in the above example, we can create the following program to trigger the issue:
```python
import parso

grammar = parso.load_grammar()
module = parso.parse("\\\n#")
if module is not None:
grammar._get_normalizer_issues(module
```

The traceback is:
```
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 42, in visit
children = node.children
AttributeError: 'EndMarker' object has no attribute 'children'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./reproducer.py", line 6, in
grammar._get_normalizer_issues(module)
File "/usr/local/lib/python3.8/site-packages/parso/grammar.py", line 203, in _get_normalizer_issues
normalizer.walk(node)
File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 36, in walk
value = self.visit(node)
File "/usr/local/lib/python3.8/site-packages/parso/python/errors.py", line 408, in visit
return super().visit(node)
File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 47, in visit
return ''.join(self.visit(child) for child in children)
File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 47, in
return ''.join(self.visit(child) for child in children)
File "/usr/local/lib/python3.8/site-packages/parso/python/errors.py", line 408, in visit
return super().visit(node)
File "/usr/local/lib/python3.8/site-packages/parso/normalizer.py", line 44, in visit
return self.visit_leaf(node)
File "/usr/local/lib/python3.8/site-packages/parso/python/pep8.py", line 350, in visit_leaf
self._visit_part(part, part.create_spacing_part(), leaf)
File "/usr/local/lib/python3.8/site-packages/parso/python/pep8.py", line 441, in _visit_part
if len(indentation) > len(n.indentation):
AttributeError: 'NoneType' object has no attribute 'indentation'
```

This was found by way of OSS-Fuzz and the set up here: https://github.com/google/oss-fuzz/tree/master/projects/parso If you find this issue helpful then it would be great to have maintainer emails in the project.yaml to receive notifications of bug reports, which contain all of the details similar to what I posted above.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.