MozillaSecurity / MozillaSecurity/FuzzManager

[FTB] UnicodeDecoderError thrown when matching non-ascii logs against unicode signatures

Open
#452 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug FTB
Dominant language
Python
Stars
208
Forks
48
Avg merge
23h 5m
Merged PRs (30d)
2

Description

Following seen during fuzzing:

---CUT---
    cache_sig_file, cache_metadata = collector.search(crash_info)
  File "/usr/local/lib/python2.7/dist-packages/Reporter/Reporter.py", line 58, in decorator
    return wrapped(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/Collector/Collector.py", line 183, in search
    if crashSig.matches(crashInfo):
  File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/CrashSignature.py", line 104, in matches
    if not symptom.matches(crashInfo):
  File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/Symptom.py", line 128, in matches
    if self.output.matches(line):
  File "/usr/local/lib/python2.7/dist-packages/FTB/Signatures/Matchers.py", line 67, in matches
    return self.value in val
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa6 in position 215: ordinal not in range(128)

I've got it down to a test that reproduces it:

# encoding=utf-8
class SignatureNonAsciiInStderr(unittest.TestCase):
    def runTest(self):
        config = ProgramConfiguration("test", "x86-64", "linux")
        log = [b"\xA6\n"]
        testSig = CrashSignature('''{
                "symptoms": [
                {
                    "type": "output",
                    "value": "ä"
                }
            ]
        }''')
        crashInfoPos = CrashInfo.fromRawCrashData([], [], config, auxCrashData=log)

        # Check that this doesn't match
        self.assertFalse(testSig.matches(crashInfoPos))

I think the right thing to do is force logs to be Unicode on submission. Doing .decode("cp437") on the log line makes the test pass. Logs should be textual, and if they can't be represented as Unicode, it should be up the reporter to ignore those errors. FuzzManager should also enforce the type that it expects.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with FTB/Signatures/Matchers.py and trace the log path through Symptom.py, CrashSignature.py, and CrashInfo.fromRawCrashData as shown in the traceback. Run the inline SignatureNonAsciiInStderr reproduction first. Done means matching non-ASCII signatures against non-ASCII log bytes no longer raises UnicodeDecodeError and the provided case remains unmatched.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.