Unexpected behaviour in Glassfish 7.1

In my institution we want to update from Glassfish 7.0.6 to 7.1. We developed our own Jakarta Authentication and Authorization modules which work with out any issues in 7.0.6. When we configure them in 7.1, they are not working correctly. They can be configured via Glassfish console correctly, but when we try to access an authentication protected webapplication after a reboot, the server replies with a 500 error message and in the log we can find the following stack trace:

Jakarta Authentication: Exception during validateRequest
java.lang.ClassCastException: class org.apache.catalina.connector.RequestFacade cannot be cast to class jakarta.xml.soap.SOAPMessage (org.apache.catalina.connector.RequestFacade is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @36454a6c; jakarta.xml.soap.SOAPMessage is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @27f4b7a0)
at com.sun.enterprise.security.webservices.WebServicesDelegateImpl.getAuthContextID(WebServicesDelegateImpl.java:91)
at com.sun.enterprise.security.ee.JavaEESecurityLifecycle.lambda$initializeJakartaAuthentication$0(JavaEESecurityLifecycle.java:103)
at org.glassfish.epicyro.config.module.config.GFAuthConfig.getAuthContextID(GFAuthConfig.java:112)
at org.glassfish.epicyro.services.BaseAuthenticationService.getServerAuthContext(BaseAuthenticationService.java:141)
at com.sun.web.security.RealmAdapter.validate(RealmAdapter.java:1272)
at com.sun.web.security.RealmAdapter.invokeAuthenticateDelegate(RealmAdapter.java:449)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:424)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:530)
at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:508)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:69)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:121)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:293)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:186)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:429)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:143)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:174)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:153)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:196)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
at java.base/java.lang.Thread.run(Thread.java:833)

It is important to note that this happens after a reboot of Glassfish. However, If we log into the console, everything starts working fine.

We debugged epicyro's code and found that the difference is that when Glassfish is restarted, in the constructor to GFAuthConfig a null reference is passed as the properties param. After logging into the console the constructor gets called with a non null reference in the properties param. This changes if the authConextIdGenerator function in Eleos Security Provider is called or not as shown below.

public GFAuthConfig(Map properties, ModuleConfigurationManager moduleConfigurationManager, AuthConfigProvider authConfigProvider, String messageLayer, String appContext, CallbackHandler handler, String authModuleType) {
this.properties = properties;
this.moduleConfigurationManager = moduleConfigurationManager;
this.authConfigProvider = authConfigProvider;
this.messageLayer = messageLayer;
this.appContext = appContext;
this.handler = handler != null ? handler : AuthMessagePolicy.getDefaultCallbackHandler();
this.authModuleType = authModuleType;

if (this.properties == null) {
Provider provider = Security.getProvider("EleosProvider");
if (provider != null) {
this.properties = new HashMap<>();
this.properties.put("authContextIdGenerator", provider.get("authContextIdGenerator"));
this.properties.put("soapPolicyGenerator", provider.get("soapPolicyGenerator"));
}
}
}

Can you point us to which Glassfish classes create the instances of GFAuthConfig so we can try to fix this issue?

Thanks in advance.
Alberto

Start by comparing the reboot and console paths in the stack trace, beginning with GFAuthConfig and its callers at JavaEESecurityLifecycle.java:103 and GFAuthConfig.java:112. Trace the creation path into WebServicesDelegateImpl.java:91 and reproduce the null-properties case after reboot. Done means identifying which GlassFish class creates the differing GFAuthConfig instances and documenting or fixing the cause of the ClassCastException.

Written by the indexing model from the issue text.