MozillaSecurity / MozillaSecurity/FuzzManager
[FTB] UnicodeDecoderError thrown when matching non-ascii logs against unicode signatures
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 208
- Forks
- 48
- Avg merge
- 23h 5m
- Merged PRs (30d)
- 2
Description
Following seen during fuzzing:
---CUT---
cache_sig_file, cache_metadata = collector.search(crash_info)
File "/usr/local/lib/python2.7/dist-packages/Reporter/Reporter.py", line 58, in decorator
return wrapped(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Collector/Collector.py", line 183, in search
if crashSig.matches(crashInfo):
File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/CrashSignature.py", line 104, in matches
if not symptom.matches(crashInfo):
File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/Symptom.py", line 128, in matches
if self.output.matches(line):
File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/Matchers.py", line 67, in matches
return self.value in val
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 215: ordinal not in range(128)
I've got it down to a test that reproduces it:
# encoding=utf-8
class SignatureNonAsciiInStderr(unittest.TestCase):
def runTest(self):
config = ProgramConfiguration("test", "x86-64", "linux")
log = [b"\xA6\n"]
testSig = CrashSignature('''{
"symptoms": [
{
"type": "output",
"value": "ä"
}
]
}''')
crashInfoPos = CrashInfo.fromRawCrashData([], [], config, auxCrashData=log)
# Check that this doesn't match
self.assertFalse(testSig.matches(crashInfoPos))
I think the right thing to do is force logs to be Unicode on submission. Doing .decode("cp437") on the log line makes the test pass. Logs should be textual, and if they can't be represented as Unicode, it should be up the reporter to ignore those errors. FuzzManager should also enforce the type that it expects.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with FTB/Signatures/Matchers.py and trace the log path through Symptom.py, CrashSignature.py, and CrashInfo.fromRawCrashData as shown in the traceback. Run the inline SignatureNonAsciiInStderr reproduction first. Done means matching non-ASCII signatures against non-ASCII log bytes no longer raises UnicodeDecodeError and the provided case remains unmatched.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100