UnicodeDecodeError in HTMLFormatter.quote with extended ASCII characters

Open
#17 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
devtools

Research direction

Start by reproducing the extended-ASCII case in weberror/formatter.py through HTMLFormatter.quote, using the Python 2.7 example in the issue. Trace how Flask passes the value and determine the expected encoding contract; done means the traceback renders without UnicodeDecodeError and the behavior is covered by an appropriate regression test.

Written by the indexing model from the issue text.

Description

I am unable to see tracebacks for a character encoding error in a Flask app I am debugging. Flask uses weberror to show tracebacks and there appears to be a character encoding issue somewhere along the way.

Either the character encoding issue is in weberror as illustrated by the test case below, or weberror has an implied contract on the encoding it expects and this is not being respected by Flask.

$ pip show weberror

---
Metadata-Version: 2.0
Name: WebError
Version: 0.13.1
from weberror import formatter

hf = formatter.HTMLFormatter()

#s = "<Request 'http://example.com?abc=def\xc5' [GET]>"
s = "\xc5"
hf.quote(s)
Traceback (most recent call last):
  File "weberror_test_case.py", line 7, in <module>
    hf.quote(s)
  File "/home/vagrant/envs/web/local/lib/python2.7/site-packages/weberror/formatter.py", line 296, in quote
    s = s.encode('latin1', 'htmlentityreplace')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 0: ordinal not in range(128)

Quick hack fix weberror/formatter.py:

class HTMLFormatter(TextFormatter):

    def quote(self, s):
        if isinstance(s, str) and hasattr(self, 'frame'):
            s = s.decode(self.frame.source_encoding, 'replace')
        #s = s.encode('latin1', 'htmlentityreplace')
        s = s.decode('latin1').encode('latin1', 'htmlentityreplace')
        return html_quote(s)

Adding the .decode('latin1') works for my purposes in that I'm now able to see tracebacks in Flask.

It's not clear to me what the expected encoding is at this point so I don't know whether this would be a correct fix for the general case, or even whether this error is expected behavior (perhaps Flask should be decoding before passing to weberror).

Dominant language
Python
Stars
17
Forks
18
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Pylons/weberror

All issues in Pylons/weberror

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.