playframework / playframework/play-ws

Unable to send requests with literal '+' character in URI query part

Open
#287 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status:backlog
Dominant language
Scala
Stars
224
Forks
92
Avg merge
1d 19h
Merged PRs (30d)
28

Description

I want to send a request to a remote HTTP service which has some strict encoding rules.

For example, HTTP requests to http://example.com/path?query+string are working, but requests to http://example.com/path?query%2Bstring are not working.

Unfortunately, Play WS always encodes the URL as http://example.com/path?query%2Bstring without the option to skip unnecessary encoding or to provide a raw query string.

Play WS Version

Play WS 2.6.20

$ sbt dependencyList|grep -E 'play-(ahc|ws)|asynchttp'
[info] com.typesafe.play:play-ahc-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ahc-ws_2.12:2.6.20
[info] com.typesafe.play:play-ws-standalone-json_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone-xml_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ws_2.12:2.6.20
[info] com.typesafe.play:shaded-asynchttpclient:1.1.10
API (Scala / Java / Neither / Both)

Java

Operating System

MacOS 10.13.6

$ uname -rspv
Darwin 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 i386
JDK
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
Expected Behavior
  1. Create WSRequest with valid URL via WSClient#url(String), for example http://example.com/path?query+string.
  2. Execute HTTP request via WSRequest#get().
  3. HTTP request is being sent to the URL provided to WSRequest (http://example.com/path?query+string).
Actual Behavior
  1. Create WSRequest with valid URL via WSClient#url(String), for example http://example.com/path?query+string.
  2. Execute HTTP request via WSRequest#get().
  3. HTTP request is being sent to an incorrectly (or too eagerly) encoded URL (http://example.com/path?query%2Bstring).
Reproducible Test Case
import com.github.tomakehurst.wiremock.junit.WireMockRule;
import org.junit.Rule;
import org.junit.Test;
import play.libs.ws.WSClient;
import play.libs.ws.WSRequest;
import play.libs.ws.WSResponse;
import play.libs.ws.ahc.AhcWSClient;
import play.shaded.ahc.org.asynchttpclient.DefaultAsyncHttpClient;

import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.junit.Assert.assertEquals;

public class WSRequestQueryParametersTest {
    @Rule
    public WireMockRule wireMockRule = new WireMockRule();

    @Test
    public void uriEncode() throws Exception {
        WSClient wsClient = new AhcWSClient(new DefaultAsyncHttpClient(), null);
        String requestUrl = wireMockRule.url("/path") + "?the+plus+must+remain";
        stubFor(get(urlEqualTo("/path?the+plus+must+remain")).willReturn(aResponse()
                .withStatus(200)
                .withBody("OK")));

        WSRequest wsRequest = wsClient.url(requestUrl);
        String url = wsRequest.getUrl();
        assertEquals("Request URL and WSRequest.Url should be identical", requestUrl, url);

        WSResponse wsResponse = wsRequest.get().toCompletableFuture().get();
        assertEquals(200, wsResponse.getStatus());

        verify(getRequestedFor(urlMatching("/path")).withQueryParam("the+plus+must+remain", equalTo("")));
    }
}
2018-10-26 15:31:57.070 +0200 [] [INFO] Logging initialized @1786ms to org.eclipse.jetty.util.log.Slf4jLog  
2018-10-26 15:31:57.248 +0200 [] [INFO] jetty-9.4.12.v20180830; built: 2018-08-30T13:59:14.071Z; git: 27208684755d94a92186989f695db2d7b21ebc51; jvm 1.8.0_181-b13  
2018-10-26 15:31:57.278 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@5b218417{/__admin,null,AVAILABLE}  
2018-10-26 15:31:57.281 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@413d1baf{/,null,AVAILABLE}  
2018-10-26 15:31:57.325 +0200 [] [INFO] Started NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}  
2018-10-26 15:31:57.326 +0200 [] [INFO] Started @2045ms  
2018-10-26 15:31:58.032 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.AdminRequestHandler. Normalized mapped under returned 'null'  
2018-10-26 15:31:58.379 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.StubRequestHandler. Normalized mapped under returned 'null'  
2018-10-26 15:31:58.395 +0200 [] [ERROR] 
                                               Request was not matched
                                               =======================

-----------------------------------------------------------------------------------------------------------------------
| Closest stub                                             | Request                                                  |
-----------------------------------------------------------------------------------------------------------------------
                                                           |
GET                                                        | GET
/path?the+plus+must+remain                                 | /path?the%2Bplus%2Bmust%2Bremain                    <<<<< URL does not match
                                                           |
                                                           |
-----------------------------------------------------------------------------------------------------------------------
  
2018-10-26 15:31:58.575 +0200 [] [INFO] Stopped NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}  
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@413d1baf{/,null,UNAVAILABLE}  
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@5b218417{/__admin,null,UNAVAILABLE}  

java.lang.AssertionError: 
Expected :200
Actual   :404

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the WSClient#url(String) and WSRequest#get() entry points, using the WSRequestQueryParametersTest reproduction in the issue. Trace where the query is encoded before the request is sent. Done means a literal '+' remains unencoded in the outgoing query and the provided WireMock test receives a 200 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala
Domain
api, networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.