spring-projects / spring-projects/spring-security
Prevent thundering herd on JWK set fetch in ReactiveRemoteJWKSource
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 9.6k
- Forks
- 6.3k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 52
Description
Expected Behavior
Concurrent requests arriving during cold start (empty JWK cache) should coalesce into a single HTTP call to the JWKS endpoint. It seems like the intension was that this already should work because of the cache() reactive operator.
IMO this would also solve the requirment for a pre fetch because it would not be necessary if there would not be massive parallel calls on cold start.
Current Behavior
Each concurrent request creates its own Mono in getJWKSet(). Since .cache() only deduplicates subscriptions to the same instance, N concurrent requests produce N separate HTTP calls, saturating the Reactor Netty connection pool. This had lead to serious problems on our side. This had made on service almost completely being stuck on instant high at cold start (which is quite common if you would scale up some new pods in kubernetes because of too high pressure of the current pods)
Context
ReactiveRemoteJWKSource.getJWKSet() - the method that fetches and caches the JWK set from the IdP's JWKS endpoint.
What are you trying to accomplish:
Ensure that concurrent JWKS fetches during cold start share a single HTTP call via atomic Mono coalescing, preventing connection pool saturation.
** solution**
I already provide a fix for it in a PR.
The tests i have created would shown that in the old code it would call jwks multiple times at once on cold start.
With the fix it's only 1 time, which is then a 'shared' Mono until it's finished.
Here is the PR:
https://github.com/spring-projects/spring-security/pull/19069
** related**
This seems to be related here, and IMO i think this would not be longer required if this thundering herd issue would be resolved:
https://github.com/spring-projects/spring-security/issues/9646
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at ReactiveRemoteJWKSource.getJWKSet() and inspect the existing cold-start fetch behavior and the tests described in the issue. Done means concurrent cold-start requests make one JWKS HTTP call and the relevant tests pass; note that PR #19069 already contains a proposed fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spring
- Domain
- security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100