jakartaee / jakartaee/websocket

TCK: WebSocket NegDep Challenge

Open
#465 19 comments 0 reactions 0 assignees View on GitHub
challenge Jakarta EE 12 Specification
Dominant language
Java
Stars
78
Forks
47
Avg merge
1h 32m
Merged PRs (30d)
1

Description

**Challenged tests**
Main Package: https://github.com/jakartaee/platform-tck/tree/main/websocket/spec-tests/src/main/java/com/sun/ts/tests/websocket/negdep

[wsc_negdep_invalidpathparamtype_srv_onclose_web](https://github.com/jakartaee/platform-tck/blob/core-tck-11.0.0/websocket/spec-tests/src/main/java/com/sun/ts/tests/websocket/negdep/invalidpathparamtype/srv/onclose/WSCClientIT.java)
[wsc_negdep_invalidpathparamtype_srv_onerror_web](https://github.com/jakartaee/platform-tck/blob/main/websocket/spec-tests/src/main/java/com/sun/ts/tests/websocket/negdep/invalidpathparamtype/srv/onerror/WSCClientIT.java)
wsc_negdep_invalidpathparamtype_srv_onmessage_web
wsc_negdep_invalidpathparamtype_srv_onopen_web
wsc_negdep_malformedpath_web
wsc_negdep_multiplepaths_web
wsc_negdep_onclose_srv_duplicate_web
wsc_negdep_onclose_srv_toomanyarguments_web
wsc_negdep_onerror_srv_duplicate_web
wsc_negdep_onerror_srv_toomanyarguments_web
wsc_negdep_onmessage_pasrv_nomoreendpoints_web
wsc_negdep_onmessage_ppsrv_nomoreendpoints_web
wsc_negdep_onmessage_srv_binarybytebufferint_web
wsc_negdep_onmessage_srv_binaryinputstreamboolean_web
wsc_negdep_onmessage_srv_binaryinputstreamboolean_web
wsc_negdep_onmessage_srv_binarynodecoder_web
wsc_negdep_onmessage_srv_pongboolean_web
wsc_negdep_onmessage_srv_pongduplicate_web
wsc_negdep_onmessage_srv_textbigdecimal_web
wsc_negdep_onmessage_srv_textduplicate_web
wsc_negdep_onmessage_srv_textnodecoder_web
wsc_negdep_onmessage_srv_textreaderboolean_web
wsc_negdep_onmessage_srv_textstringint_web
wsc_negdep_onopen_srv_duplicate_web
wsc_negdep_onopen_srv_toomanyarguments_web

**TCK Version**
[Main Branch of platform-tck/websocket](https://github.com/jakartaee/platform-tck/tree/main/websocket)

**Description**
This is partly a challenge and partly a request for improvement. These web socket tests listed above have moved to use the Arquillian framework. They invoke the application and then expect an exception to occur, as these are negative tests (some misconfiguration exists in the app).


While testing on Liberty, we have encountered various errors when trying to get these tests to run. By design, these apps trigger DeploymentException, which causes them to not start in the server. As a result, Arquillian and any related plugins we use cannot handle the app (since it doesn’t exist for us). After making a few fixes on our side, the last error we’ve encountered is related to the URL: 



```
java.lang.RuntimeException: Could not lookup value for field protected java.net.URL com.sun.ts.tests.websocket.common.client.WebSocketCommonClient.url
at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.enrich(ArquillianResourceTestEnricher.java:68)
at org.jboss.arquillian.test.impl.TestInstanceEnricher.enrich(TestInstanceEnricher.java:51)
at org.jboss.arquillian.container.test.impl.ClientTestInstanceEnricher.enrich(ClientTestInstanceEnricher.java:48)
…

Caused by: java.lang.RuntimeException: All Providers for type class java.net.URL returned a null value: [org.jboss.arquillian.container.test.impl.enricher.resource.URLResourceProvider@573d07e6]
at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.lookup(ArquillianResourceTestEnricher.java:126)
at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.enrich(ArquillianResourceTestEnricher.java:66)

```

We’re unsure if any other implementations have been tested yet, but we would expect similar issues to occur. Arquillian is trying to do some Test enrichment, but due to the deployment exception, there is no available URL for these tests.

We propose to remove the invoke all in these tests: 




```
try {
invoke(false);
} catch (Exception tfe) {
// DeploymentException
}


```

and instead rely on the Arquillian’s ShouldThrowException annotation. This has the added bonus of being able to explicitly check for DeploymentException, instead of just catching Exception as is currently done. This is similar to the CDI TCK’s approach, as linked [here](https://github.com/jakartaee/cdi-tck/blob/4.1/impl/src/main/java/org/jboss/cdi/tck/tests/invokers/lookup/LookupAmbiguousTest.java#L52) and [here](https://github.com/jakartaee/platform-tck/blob/6641e0bb961a3e5548640ac78d2ce0ea95b9c256/cdi-ee-tck/tck/src/main/java/org/jboss/cdi/tck/tests/lookup/injectionpoint/broken/not/bean/InjectionPointTest.java#L24).

Some extra handling would also be needed to verify that any previously deployed endpoints are removed from service. A draft PR can be provided.

Of if anyone has a workaround for the URL issue above, we’d be happy to accept other suggestions.

**Additional context**
None

Contributor guide

Open the contributing guide

Research direction

Start in websocket/spec-tests/src/main/java/com/sun/ts/tests/websocket/negdep and inspect the listed WSCClientIT files, especially the invalidpathparamtype onclose and onerror cases. Compare their negative-test handling with the linked CDI TCK examples and run the WebSocket negdep tests. Done means the tests avoid URL enrichment failures, explicitly verify DeploymentException, and account for removal of previously deployed endpoints.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.