ruby / ruby/webrick

Unicode handling in header location

Open
#110 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
327
Forks
112
Avg merge
4h 2m
Merged PRs (30d)
1

Description

webrick doesn't handle Unicode in HTTP location header, eg. redirection to an URL like http://dxczjjuegupb.cloudfront.net/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg.

[2023-02-17 16:41:33] ERROR URI::InvalidURIError: URI must be ascii only "http://dxczjjuegupb.cloudfront.net/wp-content/uploads/2017/10/\u041E\u0443\u044D\u043D-\u041C\u044D\u0442\u044C\u044E\u0441.jpg"                                                                                                              
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:20:in `split'                                                                                                                                                
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:71:in `parse'                                                                                                                                                
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:111:in `convert_to_uri'                                                                                                                                      
        /usr/local/lib/ruby/3.2.0/uri/generic.rb:1110:in `merge'                                                                                                                                                     
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpresponse.rb:320:in `setup_header'                                                                                                                       
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpresponse.rb:240:in `send_response'                                                                                                                      
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpserver.rb:112:in `run'                                                                                                                                  
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/server.rb:310:in `block in start_thread'

The following code is responsible:

https://github.com/ruby/webrick/blob/55101863dcf5f56ae763258a6d0919e66f76fa5f/lib/webrick/httpresponse.rb#L320

This is because methods such as URI.parse or here URI.merge only handles ASCII.

uri = URI.parse('http://dxczjjuegupb.cloudfront.net')
uri.merge('/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg').to_s
/home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:20:in `split': URI must be ascii only "/wp-content/uploads/2017/10/\u041E\u0443\u044D\u043D-\u041C\u044D\u0442\u044C\u044E\u0441.jpg" (URI::InvalidURIError)                                                                                                                                                                      
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:71:in `parse'                                                                                   
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:111:in `convert_to_uri'                                                                         
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/generic.rb:1110:in `merge'                                                                                        
        from (irb):9:in `<main>'                                                                                                                                                        
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'                                                                  
        from /home/noraj/.asdf/installs/ruby/3.2.0/bin/irb:25:in `load'                                                                                                                 
        from /home/noraj/.asdf/installs/ruby/3.2.0/bin/irb:25:in `<main>'

So URL or fragments should be escaped first, with CGI.escape for URL component and URI::Parser.new.escape for full URLs.

Examples in https://github.com/noraj/ctf-party/blob/master/lib/ctf_party/cgi.rb.

cf. https://stackoverflow.com/questions/46849219/ruby-uriinvalidurierror-uri-must-be-ascii-only/75487328

patched code:

uri.merge(CGI.escape('/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg')).to_s
# => "http://dxczjjuegupb.cloudfront.net/%2Fwp-content%2Fuploads%2F2017%2F10%2F%D0%9E%D1%83%D1%8D%D0%BD-%D0%9C%D1%8D%D1%82%D1%8C%D1%8E%D1%81.jpg"

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at lib/webrick/httpresponse.rb around the setup_header code linked in the issue, and reproduce the failure with a Unicode URL in the HTTP Location header. Review the URI.merge behavior and the escaping examples provided; done means a Unicode redirect no longer raises URI::InvalidURIError and produces a valid escaped Location value.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.