drizzle-team / drizzle-team/drizzle-orm

Enhancement request: add single-flight deduplication to `$withCache` to avoid cache-stampede under concurrent load

Open
#5,841 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
35.8k
Forks
1.6k
Avg merge
2d 7h
Merged PRs (30d)
4

Description

**Severity:** LOW — performance / cache-behavior enhancement; no data loss, no stale data.

---

## Description

`PgPreparedQuery.queryWithCache` in `drizzle-orm/src/pg-core/session.ts` uses a check-then-act pattern with no in-flight deduplication. Many cache implementations intentionally do not provide single-flight guarantees — that is by design at the cache layer. However, for caches used with `$withCache`, adding deduplication at the ORM layer would prevent a common cache-stampede scenario: when two or more concurrent SELECTs share the same cache key and the cache is cold, every caller independently calls `cache.get`, sees `undefined`, and fires a separate DB query. Under a burst (cold start or cache invalidation + traffic spike) this multiplies DB load proportionally to concurrency.

This is an enhancement request to add optional in-flight deduplication in the ORM's cache integration layer, which would benefit any cache backend regardless of whether the backend itself offers this guarantee.

## Affected code

```typescript
113 | if (this.queryMetadata.type === 'select') {
114 | const fromCache = await this.cache.get(
115 | this.cacheConfig.tag ?? await hashQuery(queryString, params), ...
119 | );
120 | if (fromCache === undefined) {
123 | result = await query(); // <-- every concurrent caller reaches here independently
128 | await this.cache.put(
129 | this.cacheConfig.tag ?? await hashQuery(queryString, params),
130 | result, ...
131 | );
132 | return result;
133 | }
134 | return fromCache as unknown as T;
```

## Suggested improvement

```typescript
private readonly inFlight = new Map>();

// miss branch:
if (fromCache === undefined) {
const cacheKey = this.cacheConfig.tag ?? await hashQuery(queryString, params);
let inflight = this.inFlight.get(cacheKey);
if (!inflight) {
inflight = query().then(async (result) => {
await this.cache.put(cacheKey, result, ...);
return result;
}).finally(() => { this.inFlight.delete(cacheKey); });
this.inFlight.set(cacheKey, inflight);
}
return inflight as Promise;
}
```

### PoC code

```js
const cacheStore = new Map();
const callLog = [];
async function cacheGet(key) { await Promise.resolve(); return cacheStore.get(key); }
async function cachePut(key, value) { await Promise.resolve(); cacheStore.set(key, value); }
async function dbQuery(label) { callLog.push(`DB_CALL:${label}`); await new Promise((r) => setTimeout(r, 10)); return [{ id: 1 }]; }
async function queryWithCache(key, queryFn) {
const fromCache = await cacheGet(key);
if (fromCache === undefined) { const result = await queryFn(); await cachePut(key, result); return result; }
return fromCache;
}
await Promise.all([
queryWithCache('select_users_all', () => dbQuery('concurrent-A')),
queryWithCache('select_users_all', () => dbQuery('concurrent-B')),
]);
console.log('DB calls fired:', callLog);
if (callLog.length === 2) { console.log('BUG_CONFIRMED'); }
```

### PoC verbatim output

```
DB calls fired: [ 'DB_CALL:concurrent-A', 'DB_CALL:concurrent-B' ]
STAMPEDE CONFIRMED: both concurrent SELECTs missed the cache and fired a duplicate DB query.
BUG_CONFIRMED
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.