drizzle-team / drizzle-team/drizzle-orm
Enhancement request: add single-flight deduplication to `$withCache` to avoid cache-stampede under concurrent load
- Dominant language
- TypeScript
- Stars
- 35.8k
- Forks
- 1.6k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 4
Description
**Severity:** LOW — performance / cache-behavior enhancement; no data loss, no stale data.
---
## Description
`PgPreparedQuery.queryWithCache` in `drizzle-orm/src/pg-core/session.ts` uses a check-then-act pattern with no in-flight deduplication. Many cache implementations intentionally do not provide single-flight guarantees — that is by design at the cache layer. However, for caches used with `$withCache`, adding deduplication at the ORM layer would prevent a common cache-stampede scenario: when two or more concurrent SELECTs share the same cache key and the cache is cold, every caller independently calls `cache.get`, sees `undefined`, and fires a separate DB query. Under a burst (cold start or cache invalidation + traffic spike) this multiplies DB load proportionally to concurrency.
This is an enhancement request to add optional in-flight deduplication in the ORM's cache integration layer, which would benefit any cache backend regardless of whether the backend itself offers this guarantee.
## Affected code
```typescript
113 | if (this.queryMetadata.type === 'select') {
114 | const fromCache = await this.cache.get(
115 | this.cacheConfig.tag ?? await hashQuery(queryString, params), ...
119 | );
120 | if (fromCache === undefined) {
123 | result = await query(); // <-- every concurrent caller reaches here independently
128 | await this.cache.put(
129 | this.cacheConfig.tag ?? await hashQuery(queryString, params),
130 | result, ...
131 | );
132 | return result;
133 | }
134 | return fromCache as unknown as T;
```
## Suggested improvement
```typescript
private readonly inFlight = new Map>();
// miss branch:
if (fromCache === undefined) {
const cacheKey = this.cacheConfig.tag ?? await hashQuery(queryString, params);
let inflight = this.inFlight.get(cacheKey);
if (!inflight) {
inflight = query().then(async (result) => {
await this.cache.put(cacheKey, result, ...);
return result;
}).finally(() => { this.inFlight.delete(cacheKey); });
this.inFlight.set(cacheKey, inflight);
}
return inflight as Promise;
}
```
### PoC code
```js
const cacheStore = new Map();
const callLog = [];
async function cacheGet(key) { await Promise.resolve(); return cacheStore.get(key); }
async function cachePut(key, value) { await Promise.resolve(); cacheStore.set(key, value); }
async function dbQuery(label) { callLog.push(`DB_CALL:${label}`); await new Promise((r) => setTimeout(r, 10)); return [{ id: 1 }]; }
async function queryWithCache(key, queryFn) {
const fromCache = await cacheGet(key);
if (fromCache === undefined) { const result = await queryFn(); await cachePut(key, result); return result; }
return fromCache;
}
await Promise.all([
queryWithCache('select_users_all', () => dbQuery('concurrent-A')),
queryWithCache('select_users_all', () => dbQuery('concurrent-B')),
]);
console.log('DB calls fired:', callLog);
if (callLog.length === 2) { console.log('BUG_CONFIRMED'); }
```
### PoC verbatim output
```
DB calls fired: [ 'DB_CALL:concurrent-A', 'DB_CALL:concurrent-B' ]
STAMPEDE CONFIRMED: both concurrent SELECTs missed the cache and fired a duplicate DB query.
BUG_CONFIRMED
```
Contributor guide
Assessment
This issue has not been assessed yet.