pingcap / pingcap/tidb

mview: COMPLETE DELTA APPLY can hit [executor:8175] under a 24 GiB session quota when catching up a large delta, while a later run with a much smaller lag succeeds

Open
#67,603 0 comments 0 reactions 0 assignees View on GitHub
component/mview feature/developing severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

Use a large dataset with:

- one large base table,
- one target materialized view refreshed manually,
- one peer materialized view on the same source that auto-refreshes every 2 minutes and can be used as a reference point.

The target materialized view had an available TiFlash replica:

- `replica_count = 1`
- `available = 1`
- `progress = 1`

Then run:

```sql
SET SESSION tidb_mem_quota_query = 25769803776;
REFRESH MATERIALIZED VIEW . COMPLETE DELTA APPLY;
```

At the failed attempt, the exact row counts were:

- base table: `2,744,539,172`
- auto-refreshed reference materialized view: `495,477,589`
- target materialized view: `485,240,069`

So the target materialized view lagged the reference materialized view by `10,237,520` rows.

### 2. What did you expect to see? (Required)

I expected `COMPLETE DELTA APPLY` to succeed under a `24 GiB` session query memory quota for this scale of catch-up work.

At minimum, if the real limitation is that the pending delta is too large, I expected a more actionable behavior or clearer indication than a generic single-query memory-limit error.

### 3. What did you see instead (Required)

The refresh failed after `887.757610s` with:

```text
ERROR 8175 (HY000): Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query.
```

Internal refresh history recorded:

```text
[executor:8175]Your query has been cancelled due to exceeding the allowed memory limit for a single SQL query. Please try narrowing your query scope or increase the tidb_mem_quota_query limit and try again.[conn=0]
```

Failed run details:

- session quota: `24 GiB`
- start: `2026-04-08 01:20:26.441024 UTC`
- end: `2026-04-08 01:35:14.198634 UTC`
- duration: `887.757610s`
- base table rows: `2,744,539,172`
- auto-refreshed reference materialized view rows: `495,477,589`
- target materialized view rows: `485,240,069`
- lag between reference materialized view and target materialized view: `10,237,520`

As a reference point only, a later `COMPLETE DELTA APPLY` run under the same `24 GiB` session quota succeeded when the lag was much smaller:

- start: `2026-04-08 03:04:42.147039 UTC`
- end: `2026-04-08 03:23:39.034120 UTC`
- duration: `1136.887081s`
- base table rows: `2,745,093,592`
- auto-refreshed reference materialized view rows: `496,028,469`
- target materialized view rows: `495,506,269`
- lag between reference materialized view and target materialized view: `522,200`

So the successful reference run had a lag about `19.6x` smaller than the failed run (`10,237,520` vs `522,200`).

This makes the issue look strongly related to pending-delta size rather than only overall table size.

### 4. What is your TiDB version? (Required)

```sql
Release Version: v8.5.4
Edition: Enterprise
Git Commit Hash: 8907b97ad28ec3d14412a0a8d48de1d08e46baad
Git Branch: heads/refs/tags/v8.5.4
UTC Build Time: 2026-04-05 08:48:00
GoVersion: go1.23.6
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Enterprise Extension Commit Hash: 7d43ff65ebc145bd63fa84cb368f8775be906998
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.