spring-projects / spring-projects/spring-security

Prevent thundering herd on JWK set fetch in ReactiveRemoteJWKSource

Open
#19,068 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage type: enhancement
Dominant language
Java
Stars
9.6k
Forks
6.3k
Avg merge
2d 11h
Merged PRs (30d)
52

Description

Expected Behavior

Concurrent requests arriving during cold start (empty JWK cache) should coalesce into a single HTTP call to the JWKS endpoint. It seems like the intension was that this already should work because of the cache() reactive operator.

IMO this would also solve the requirment for a pre fetch because it would not be necessary if there would not be massive parallel calls on cold start.

Current Behavior

Each concurrent request creates its own Mono in getJWKSet(). Since .cache() only deduplicates subscriptions to the same instance, N concurrent requests produce N separate HTTP calls, saturating the Reactor Netty connection pool. This had lead to serious problems on our side. This had made on service almost completely being stuck on instant high at cold start (which is quite common if you would scale up some new pods in kubernetes because of too high pressure of the current pods)

Context
ReactiveRemoteJWKSource.getJWKSet() - the method that fetches and caches the JWK set from the IdP's JWKS endpoint.

What are you trying to accomplish:
Ensure that concurrent JWKS fetches during cold start share a single HTTP call via atomic Mono coalescing, preventing connection pool saturation.

** solution**

I already provide a fix for it in a PR.
The tests i have created would shown that in the old code it would call jwks multiple times at once on cold start.
With the fix it's only 1 time, which is then a 'shared' Mono until it's finished.

Here is the PR:
https://github.com/spring-projects/spring-security/pull/19069

** related**
This seems to be related here, and IMO i think this would not be longer required if this thundering herd issue would be resolved:
https://github.com/spring-projects/spring-security/issues/9646

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at ReactiveRemoteJWKSource.getJWKSet() and inspect the existing cold-start fetch behavior and the tests described in the issue. Done means concurrent cold-start requests make one JWKS HTTP call and the relevant tests pass; note that PR #19069 already contains a proposed fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.