jakartaee / jakartaee/security

TCK challenge for adding support for running Security 3.0 signature TCK tests on Java 21

Open
#297 15 comments 1 reaction 0 assignees View on GitHub
accepted challenge
Dominant language
Java
Stars
66
Forks
43
PR merge metrics
No merged PRs in 30d

Description

When running the https://www.eclipse.org/downloads/download.php?file=/jakartaee/security/3.0/jakarta-security-tck-3.0.1.zip tests on Java 21, the following signature test failures occur due to Java 21 added a few new JDK methods:

> 12-01-2023 11:32:57:********** Status Report 'jakarta.security.enterprise.identitystore.openid' **********
12-01-2023 11:32:57:SignatureTest report
Base version: 3.0.0
Tested version: 3.0.0
Check mode: src [throws normalized]
Constant checking: on
>
> Added Superclasses or Superinterfaces
> -------------------------------------
>
>jakarta.security.enterprise.identitystore.openid.Scope: interface java.util.SequencedCollection
jakarta.security.enterprise.identitystore.openid.Scope: interface java.util.SequencedSet
>
>Added Methods
> -------------
>
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.getFirst()
jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.getLast()
jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.removeFirst()
jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.removeLast()
jakarta.security.enterprise.identitystore.openid.Scope: method public java.util.SequencedSet java.util.LinkedHashSet.reversed()
jakarta.security.enterprise.identitystore.openid.Scope: method public static <%0 extends java.lang.Object> java.util.HashSet<{%%0}> java.util.HashSet.newHashSet(int)
jakarta.security.enterprise.identitystore.openid.Scope: method public static <%0 extends java.lang.Object> java.util.LinkedHashSet<{%%0}> java.util.LinkedHashSet.newLinkedHashSet(int)
jakarta.security.enterprise.identitystore.openid.Scope: method public void java.util.LinkedHashSet.addFirst(java.lang.String)
jakarta.security.enterprise.identitystore.openid.Scope: method public void java.util.LinkedHashSet.addLast(java.lang.String)
>
> 12-01-2023 11:32:57:********** Package 'jakarta.security.enterprise.identitystore.openid' - FAILED (STATIC MODE) **********

> 12-01-2023 11:32:58:********** Status Report 'jakarta.security.enterprise.identitystore.openid' **********
>
> 12-01-2023 11:32:58:SignatureTest report
> Base version: 3.0.0
> Tested version: 3.0.0
> Check mode: src [throws normalized]
> Constant checking: on
>
>
> Added Superclasses or Superinterfaces
> -------------------------------------
>
> jakarta.security.enterprise.identitystore.openid.Scope: interface java.util.SequencedCollection
> jakarta.security.enterprise.identitystore.openid.Scope: interface java.util.SequencedSet
>
> Added Methods
> -------------
>
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.getFirst()
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.getLast()
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.removeFirst()
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.lang.String java.util.LinkedHashSet.removeLast()
> jakarta.security.enterprise.identitystore.openid.Scope: method public java.util.SequencedSet java.util.LinkedHashSet.reversed()
> jakarta.security.enterprise.identitystore.openid.Scope: method public static <%0 extends java.lang.Object> java.util.HashSet<{%%0}> java.util.HashSet.newHashSet(int)
> jakarta.security.enterprise.identitystore.openid.Scope: method public static <%0 extends java.lang.Object> java.util.LinkedHashSet<{%%0}> java.util.LinkedHashSet.newLinkedHashSet(int)
> jakarta.security.enterprise.identitystore.openid.Scope: method public void java.util.LinkedHashSet.addFirst(java.lang.String)
> jakarta.security.enterprise.identitystore.openid.Scope: method public void java.util.LinkedHashSet.addLast(java.lang.String)
>

> 12-01-2023 11:32:58:SigTestEE.signatureTest() failed!, diffs found
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.252 s <<< FAILURE! - in ee.jakarta.tck.security.signaturetest.SecurityAPISigTestIT
> [ERROR] ee.jakarta.tck.security.signaturetest.SecurityAPISigTestIT.signatureTest Time elapsed: 6.074 s <<< ERROR!
> ee.jakarta.tck.security.signaturetest.SigTestEE$Fault: SigTestEE.signatureTest() failed!, diffs found
> at security.tck.sigtest@3.0.0/ee.jakarta.tck.security.signaturetest.SecurityAPISigTestIT.signatureTest(SecurityAPISigTestIT.java:355)
>

There are two possible fixes, the easiest one line change would be to exclude the specific updated JDK classes on the command line when invoking the sigtest tool. I'll look into that next year.

The better change would be to update https://github.com/jtulach/netbeans-apitest to be able to avoid checking any SPEC API classes that are not in the `jakarta` namespace which I think would avoid validation of any JDK classes and instead focus only on checking `jakarta.*` classes. I'd like to do this change for Jakarta EE 11+.

Contributor guide

Open the contributing guide

Research direction

Start with ee.jakarta.tck.security.signaturetest.SecurityAPISigTestIT.java at line 355 and the sigtest invocation that produces the Java 21 failures. Then inspect the netbeans-apitest project and reproduce the Security 3.0 TCK signature test on Java 21. Done means the test no longer reports added JDK members while continuing to validate jakarta.* API signatures.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
security, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.