tornadoweb / tornadoweb/tornado

Possible unicode issues with validate_absolute_path

Open
#1,849 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

web
Dominant language
Python
Stars
22.2k
Forks
5.6k
Avg merge
3h 42m
Merged PRs (30d)
16

Description

Hi, I caught this traceback info in my server logs:

    Uncaught exception GET /s/chat.html%E2%80%8F (**.**.**.**)
    HTTPServerRequest(protocol='http', host='www.****.com', method='GET', uri='/s/chat.html%E2%80%8F', version='HTTP/1.1', remote_ip='**.**.**.**', headers={'X-Real-Ip': '**.**.**.**, 'Accept-Language': 'he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4', 'Host': 'www.****.com', 'Connection': 'upgrade', 'Upgrade-Insecure-Requests': '1', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36'})
    Traceback (most recent call last):
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/web.py", line 1445, in _execute
            result = yield result
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/gen.py", line 1008, in run
            value = future.result()
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/concurrent.py", line 232, in result
            raise_exc_info(self._exc_info)
        File "<string>", line 3, in raise_exc_info
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/gen.py", line 282, in wrapper
            yielded = next(result)
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/web.py", line 2265, in get
            self.root, absolute_path)
        File "/home/production/.virtualenvs/wchat/lib/python3.5/site-packages/tornado/web.py", line 2446, in validate_absolute_path
            if (os.path.isdir(absolute_path) and
        File "/home/production/.virtualenvs/wchat/lib/python3.5/genericpath.py", line 42, in isdir
            st = os.stat(s)
    UnicodeEncodeError: 'ascii' codec can't encode character '\u200f' in position 52: ordinal not in range(128)

(I removed the exact IP of the user and the domain names).
This error seems to happen every time I access the path /s/chat.html%E2%80%8F in my server. Note that this path (everything under /s) is server using web.StaticFileHandler.

I still haven't fully investigated this myself, though I was wondering if anyone had a similar problem. For reference, my installed Tornado version is 4.3. I am using python 3.5.1.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tornado/web.py at StaticFileHandler and validate_absolute_path, using the reported /s/chat.html%E2%80%8F request and Python 3.5.1 environment to reproduce the traceback. Inspect how the percent-encoded path reaches os.path.isdir; done means the request no longer raises UnicodeEncodeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.