playframework / playframework/play-ws
Unable to send requests with literal '+' character in URI query part
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 224
- Forks
- 92
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 28
Description
I want to send a request to a remote HTTP service which has some strict encoding rules.
For example, HTTP requests to http://example.com/path?query+string are working, but requests to http://example.com/path?query%2Bstring are not working.
Unfortunately, Play WS always encodes the URL as http://example.com/path?query%2Bstring without the option to skip unnecessary encoding or to provide a raw query string.
Play WS Version
Play WS 2.6.20
$ sbt dependencyList|grep -E 'play-(ahc|ws)|asynchttp'
[info] com.typesafe.play:play-ahc-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ahc-ws_2.12:2.6.20
[info] com.typesafe.play:play-ws-standalone-json_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone-xml_2.12:1.1.10
[info] com.typesafe.play:play-ws-standalone_2.12:1.1.10
[info] com.typesafe.play:play-ws_2.12:2.6.20
[info] com.typesafe.play:shaded-asynchttpclient:1.1.10
API (Scala / Java / Neither / Both)
Java
Operating System
MacOS 10.13.6
$ uname -rspv
Darwin 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 i386
JDK
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
Expected Behavior
- Create
WSRequestwith valid URL viaWSClient#url(String), for examplehttp://example.com/path?query+string. - Execute HTTP request via
WSRequest#get(). - HTTP request is being sent to the URL provided to
WSRequest(http://example.com/path?query+string).
Actual Behavior
- Create
WSRequestwith valid URL viaWSClient#url(String), for examplehttp://example.com/path?query+string. - Execute HTTP request via
WSRequest#get(). - HTTP request is being sent to an incorrectly (or too eagerly) encoded URL (
http://example.com/path?query%2Bstring).
Reproducible Test Case
import com.github.tomakehurst.wiremock.junit.WireMockRule;
import org.junit.Rule;
import org.junit.Test;
import play.libs.ws.WSClient;
import play.libs.ws.WSRequest;
import play.libs.ws.WSResponse;
import play.libs.ws.ahc.AhcWSClient;
import play.shaded.ahc.org.asynchttpclient.DefaultAsyncHttpClient;
import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.junit.Assert.assertEquals;
public class WSRequestQueryParametersTest {
@Rule
public WireMockRule wireMockRule = new WireMockRule();
@Test
public void uriEncode() throws Exception {
WSClient wsClient = new AhcWSClient(new DefaultAsyncHttpClient(), null);
String requestUrl = wireMockRule.url("/path") + "?the+plus+must+remain";
stubFor(get(urlEqualTo("/path?the+plus+must+remain")).willReturn(aResponse()
.withStatus(200)
.withBody("OK")));
WSRequest wsRequest = wsClient.url(requestUrl);
String url = wsRequest.getUrl();
assertEquals("Request URL and WSRequest.Url should be identical", requestUrl, url);
WSResponse wsResponse = wsRequest.get().toCompletableFuture().get();
assertEquals(200, wsResponse.getStatus());
verify(getRequestedFor(urlMatching("/path")).withQueryParam("the+plus+must+remain", equalTo("")));
}
}
2018-10-26 15:31:57.070 +0200 [] [INFO] Logging initialized @1786ms to org.eclipse.jetty.util.log.Slf4jLog
2018-10-26 15:31:57.248 +0200 [] [INFO] jetty-9.4.12.v20180830; built: 2018-08-30T13:59:14.071Z; git: 27208684755d94a92186989f695db2d7b21ebc51; jvm 1.8.0_181-b13
2018-10-26 15:31:57.278 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@5b218417{/__admin,null,AVAILABLE}
2018-10-26 15:31:57.281 +0200 [] [INFO] Started o.e.j.s.ServletContextHandler@413d1baf{/,null,AVAILABLE}
2018-10-26 15:31:57.325 +0200 [] [INFO] Started NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-10-26 15:31:57.326 +0200 [] [INFO] Started @2045ms
2018-10-26 15:31:58.032 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.AdminRequestHandler. Normalized mapped under returned 'null'
2018-10-26 15:31:58.379 +0200 [] [INFO] RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.StubRequestHandler. Normalized mapped under returned 'null'
2018-10-26 15:31:58.395 +0200 [] [ERROR]
Request was not matched
=======================
-----------------------------------------------------------------------------------------------------------------------
| Closest stub | Request |
-----------------------------------------------------------------------------------------------------------------------
|
GET | GET
/path?the+plus+must+remain | /path?the%2Bplus%2Bmust%2Bremain <<<<< URL does not match
|
|
-----------------------------------------------------------------------------------------------------------------------
2018-10-26 15:31:58.575 +0200 [] [INFO] Stopped NetworkTrafficServerConnector@31995969{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@413d1baf{/,null,UNAVAILABLE}
2018-10-26 15:31:58.577 +0200 [] [INFO] Stopped o.e.j.s.ServletContextHandler@5b218417{/__admin,null,UNAVAILABLE}
java.lang.AssertionError:
Expected :200
Actual :404
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the WSClient#url(String) and WSRequest#get() entry points, using the WSRequestQueryParametersTest reproduction in the issue. Trace where the query is encoded before the request is sent. Done means a literal '+' remains unencoded in the outgoing query and the provided WireMock test receives a 200 response.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala
- Domain
- api, networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100