Sporadic ClaimsChallengeRequiredException using client secret authentication in newer versions of SDK

Describe the bug

After upgrading the library to version 6.18.0 (or any version other than 6.12.0), we sporadically encounter ClaimsChallengeRequiredExceptions.

When this issue arises, the com.microsoft.graph.serviceclient.GraphServiceClient consistently returns ClaimsChallengeRequiredException, and this can persist for several hours before the problem resolves itself. The issue may or may not recur the following day, with occurrences ranging from twice a day to once every four days.

Sample response from the Graph API:

{ 
 "error":{
   "code":"InvalidAuthenticationToken",
   "message":"Exception of type 'Microsoft.Graph.AGS.Contracts.ClaimsChallengeRequiredException' was thrown.",
   "innerError": {"date":"2024-10-25T12:28:03", "request-id":"f9d0585e-13fc-45d5-8e04-052b9768bcc0", "client-request-id":"83e18f5d-2547-4cac-8aa1-3b11f3a8148d"}
 }
}

Downgrading to 6.12.0 will cause the problem to not appea, while other applications running 6.18.0 for the same service principal still get the error.

Expected behavior

We do not expect to get sporadic ClaimsChallengeRequiredException's

How to reproduce

In our spring boot kotlin application we have defined a spring bean for a GraphServiceClient like this

@Bean
fun graphServiceClient(): GraphServiceClient {
    return GraphServiceClient(
        ClientSecretCredentialBuilder()
            .clientId(azureProperties.clientId)
            .clientSecret(azureProperties.clientSecret)
            .tenantId(azureProperties.tenantId)
            .build(),
        SCOPES,
    )
}

We also have a health check that pings

graphServiceClient.applicationsWithAppId(azureProperties.clientId).get()

to verify that the client works.

With this we can expect the exception to be thrown at any moment / random.
The following image displays occurances of this exception the last 14 days.

SDK Version

6.18.0

Latest version known to work for scenario above?

6.12.0

Known Workarounds

We have currently two workarounds:

1. Restarting the application, which reinitializes the GraphServiceClient bean
   or
2. Wait an hour or two for the problem to disappear

Other information

When running multiple instances of the same application, using the same service principal, all instances will be affected at the same time, when this issue occurs.

Start with the GraphServiceClient bean using ClientSecretCredentialBuilder and the applicationsWithAppId health check. Compare authentication behavior between SDK versions 6.12.0 and 6.18.0, focusing on why all instances using the same service principal receive ClaimsChallengeRequiredException together. Done means the sporadic exception is reproduced or its regression is identified and a verified fix is available.