bug: concurrent `/identity/connect/token` crashes Identity on premium license cache race
- Dominant language
- C#
- Stars
- 20.1k
- Forks
- 1.8k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 75
Description
### Steps To Reproduce
1. Run a self-hosted Bitwarden lite instance (v2026.8.1) with a premium user and a valid license at `/etc/bitwarden/licenses/user/{selfHostedUserId}.json`.
2. Log in on a mobile client (reproduced with Bitwarden iOS 2026.8.0) so the client holds a refresh token.
3. Trigger two overlapping `POST /identity/connect/token` requests for that user (refresh-token grant). This happens naturally when the iOS app retries / opens in the background; it can also be done with two parallel curls using the same refresh token.
4. Observe Identity logs and the client receiving HTTP 504 / an unexpected error.
### Expected Result
Token issuance succeeds (or returns a normal OAuth error). Concurrent refresh requests for the same premium user must not crash Identity.
### Actual Result
The first overlapping request can succeed or fail; the second throws an unhandled exception while building the access token. Identity then serves 504s until the process restarts.
```
System.ArgumentException: An item with the same key has already been added. Key: {selfHostedUserId}
at System.Collections.Generic.Dictionary`2.Add(...)
at Bit.Core.Billing.Services.LicensingService.ValidateUserPremiumAsync(User user)
in /source/src/Core/Billing/Services/Implementations/LicensingService.cs:line 249
at Bit.Identity.IdentityServer.ProfileService.GetProfileDataAsync(...)
at Duende.IdentityServer.ResponseHandling.TokenResponseGenerator.ProcessRefreshTokenRequestAsync(...)
```
Follow-up requests then hit:
```
System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state.
at Bit.Core.Billing.Services.LicensingService.ValidateUserPremiumAsync(User user)
```
This is not a duplicate license file. There is a single `user/{selfHostedUserId}.json`. The dictionary key is the self-hosted user id; the `Id` inside the JSON is the cloud user id (expected).
### Screenshots or Videos
_No response_
### Additional Context
`LicensingService` keeps an in-memory cache:
```csharp
private IDictionary _userCheckCache = new Dictionary();
```
In `ValidateUserPremiumAsync` (still present on `master`):
```csharp
if (_userCheckCache.TryGetValue(user.Id, out var lastCheck))
{
// ...
}
else
{
_userCheckCache.Add(user.Id, now); // race
}
```
Two concurrent token requests for the same premium user both miss the cache and both call `.Add`. The unhandled exception aborts `/identity/connect/token`.
Suggested fix: use `ConcurrentDictionary` (or `IMemoryCache`) and `TryAdd` / indexer assignment instead of `Dictionary.Add`.
Related: https://github.com/bitwarden/server/issues/4274 (same stack traces; closed as an email/license mismatch; the race was not fixed).
### Build Version
Bitwarden lite `2026.8.1` (`ghcr.io/bitwarden/lite`). Reproduced against server tag `v2026.8.1`; the racy `_userCheckCache.Add` is still on `master`.
### Environment
Self-Hosted
### Environment Details
- Deployment: Kubernetes, Bitwarden lite (single container)
- Identity listens on `http://+:5005`
- Premium individual license (not an organization)
- PostgreSQL 18
- ref #4274
### Issue Tracking Info
- [x] I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.
Contributor guide
Research direction
Start in src/Core/Billing/Services/Implementations/LicensingService.cs, especially ValidateUserPremiumAsync and the _userCheckCache access. Reproduce the race with two parallel refresh-token requests, then verify that concurrent requests for one premium user no longer throw and that /identity/connect/token returns success or a normal OAuth error instead of 504s.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- authentication, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100