mview: COMPLETE DELTA APPLY can hit [executor:8175] under a 24 GiB session quota when catching up a large delta, while a later run with a much smaller lag succeeds
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
Use a large dataset with:
- one large base table,
- one target materialized view refreshed manually,
- one peer materialized view on the same source that auto-refreshes every 2 minutes and can be used as a reference point.
The target materialized view had an available TiFlash replica:
- `replica_count = 1`
- `available = 1`
- `progress = 1`
Then run:
```sql
SET SESSION tidb_mem_quota_query = 25769803776;
REFRESH MATERIALIZED VIEW . COMPLETE DELTA APPLY;
```
At the failed attempt, the exact row counts were:
- base table: `2,744,539,172`
- auto-refreshed reference materialized view: `495,477,589`
- target materialized view: `485,240,069`
So the target materialized view lagged the reference materialized view by `10,237,520` rows.
### 2. What did you expect to see? (Required)
I expected `COMPLETE DELTA APPLY` to succeed under a `24 GiB` session query memory quota for this scale of catch-up work.
At minimum, if the real limitation is that the pending delta is too large, I expected a more actionable behavior or clearer indication than a generic single-query memory-limit error.
### 3. What did you see instead (Required)
The refresh failed after `887.757610s` with:
```text
ERROR 8175 (HY000): Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query.
```
Internal refresh history recorded:
```text
[executor:8175]Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query. Please try narrowing your query scope or increase the tidb_mem_quota_query limit and try again.[conn=0]
```
Failed run details:
- session quota: `24 GiB`
- start: `2026-04-08 01:20:26.441024 UTC`
- end: `2026-04-08 01:35:14.198634 UTC`
- duration: `887.757610s`
- base table rows: `2,744,539,172`
- auto-refreshed reference materialized view rows: `495,477,589`
- target materialized view rows: `485,240,069`
- lag between reference materialized view and target materialized view: `10,237,520`
As a reference point only, a later `COMPLETE DELTA APPLY` run under the same `24 GiB` session quota succeeded when the lag was much smaller:
- start: `2026-04-08 03:04:42.147039 UTC`
- end: `2026-04-08 03:23:39.034120 UTC`
- duration: `1136.887081s`
- base table rows: `2,745,093,592`
- auto-refreshed reference materialized view rows: `496,028,469`
- target materialized view rows: `495,506,269`
- lag between reference materialized view and target materialized view: `522,200`
So the successful reference run had a lag about `19.6x` smaller than the failed run (`10,237,520` vs `522,200`).
This makes the issue look strongly related to pending-delta size rather than only overall table size.
### 4. What is your TiDB version? (Required)
```sql
Release Version: v8.5.4
Edition: Enterprise
Git Commit Hash: 8907b97ad28ec3d14412a0a8d48de1d08e46baad
Git Branch: heads/refs/tags/v8.5.4
UTC Build Time: 2026-04-05 08:48:00
GoVersion: go1.23.6
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Enterprise Extension Commit Hash: 7d43ff65ebc145bd63fa84cb368f8775be906998
```
Contributor guide
Assessment
This issue has not been assessed yet.