openresty / openresty/lua-nginx-module

Impossible to rewrite URI with arbitrary escape sequences like %2F

Open
#659 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
11.8k
Forks
2.1k
Avg merge
6h 1m
Merged PRs (30d)
6

Description

        location /bug {
            rewrite_by_lua      'ngx.req.set_uri("/foo%2Fbar")';
            proxy_pass          http://example.com;
        }

Fetch /bug and watch the HTTP traffic with tcpdump:

05:23:19.205157 IP 10.0.2.15.50345 > 93.184.216.34.80: Flags [P.], seq 1:106, ack 1, win 29200, length 105
        0x0000:  5254 0012 3502 0800 2719 bec2 0800 4500  RT..5...'.....E.
        0x0010:  0091 5177 4000 4006 a706 0a00 020f 5db8  ..Qw@.@.......].
        0x0020:  d822 c4a9 0050 6bdf 4ae6 c6b9 0c02 5018  ."...Pk.J.....P.
        0x0030:  7210 426d 0000 4745 5420 2f66 6f6f 2532  r.Bm..GET./foo%2
        0x0040:  3532 4662 6172 2048 5454 502f 312e 300d  52Fbar.HTTP/1.0.
        0x0050:  0a48 6f73 743a 2065 7861 6d70 6c65 2e63  .Host:.example.c
        0x0060:  6f6d 0d0a 436f 6e6e 6563 7469 6f6e 3a20  om..Connection:.
        0x0070:  636c 6f73 650d 0a55 7365 722d 4167 656e  close..User-Agen
        0x0080:  743a 2063 7572 6c2f 372e 3433 2e30 0d0a  t:.curl/7.43.0..
        0x0090:  4163 6365 7074 3a20 2a2f 2a0d 0a0d 0a    Accept:.*/*....

The outbound request is GET /foo%252Fbar HTTP/1.0; Not what I expect. Percent characters are always escaped.

Our app server expects to parse URIs by splitting on collapsed forward-slash characters, and percent-decoding each component. With this bug, there is no way for us to use a forward-slash character within a URI component.

For more fun, remove or comment the rewrite_by_lua rule, and fetch /bug/foo%2Fbar. Works as expected:

05:36:52.027511 IP 10.0.2.15.50352 > 93.184.216.34.80: Flags [P.], seq 1:108, ack 1, win 29200, length 107
        0x0000:  5254 0012 3502 0800 2719 bec2 0800 4500  RT..5...'.....E.
        0x0010:  0093 d3d3 4000 4006 24a8 0a00 020f 5db8  ....@.@.$.....].
        0x0020:  d822 c4b0 0050 911d e685 cb1e 0c02 5018  ."...P........P.
        0x0030:  7210 426f 0000 4745 5420 2f62 7567 2f66  r.Bo..GET./bug/f
        0x0040:  6f6f 2532 4662 6172 2048 5454 502f 312e  oo%2Fbar.HTTP/1.
        0x0050:  300d 0a48 6f73 743a 2065 7861 6d70 6c65  0..Host:.example
        0x0060:  2e63 6f6d 0d0a 436f 6e6e 6563 7469 6f6e  .com..Connection
        0x0070:  3a20 636c 6f73 650d 0a55 7365 722d 4167  :.close..User-Ag
        0x0080:  656e 743a 2063 7572 6c2f 372e 3433 2e30  ent:.curl/7.43.0
        0x0090:  0d0a 4163 6365 7074 3a20 2a2f 2a0d 0a0d  ..Accept:.*/*...
        0x00a0:  0a                                       .

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the nginx rewrite_by_lua path and the ngx.req.set_uri behavior shown in the reproduction, then compare it with the direct /bug/foo%2Fbar request. Done means an equivalent rewritten URI reaches the upstream as /foo%2Fbar rather than /foo%252Fbar, while preserving the existing direct-request behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua, nginx
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.