pingcap / pingcap/tidb

the session hang with copIteratorWork meet panic

Open
#38,528 0 comments 0 reactions 0 assignees View on GitHub
affects-5.4 affects-6.1 affects-6.4 may-affects-4.0 may-affects-5.0 may-affects-5.1 may-affects-5.2 may-affects-5.3 may-affects-6.0 may-affects-6.2 may-affects-6.3 severity/moderate sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
1. deploy a cluster with tikv
2. execute the sql
```sql
create table t1(a int, index(a));
create table t2(a int, index(a));
insert into t1 values(1),(2);
insert into t2 values(1),(1),(2),(2);
set @@tidb_mem_quota_query=1000;
set tidb_index_join_batch_size = 1;
explain select /*+ inl_join(t1) */ * from t1 join t2 on t1.a = t2.a;
select /*+ inl_join(t1) */ * from t1 join t2 on t1.a = t2.a;
```

### 2. What did you expect to see? (Required)
query successfully.
### 3. What did you see instead (Required)
the session was hang and no result.
```log
[2022/10/18 18:44:31.529 +08:00] [WARN] [expensivequery.go:114] [expensive_query] [cost_time=0.001389086s] [stats=t1:pseudo,t2:pseudo] [conn_id=6315680551852310935] [user=root] [database=test] [index_names="[t2:a]"] [txn_start_ts=436755463279738881] [mem_max="1008 Bytes (1008 Bytes)"] [sql="select /*+ inl_join(t1) */ * from t1 join t2 on t1.a = t2.a"]
[2022/10/18 18:44:31.529 +08:00] [ERROR] [adapter.go:150] ["execute sql panic"] [conn=6315680551852310935] [sql="select /*+ inl_join(t1) */ * from t1 join t2 on t1.a = t2.a"] [stack="github.com/pingcap/tidb/executor.(*recordSet).Next.func1\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:150\nruntime.gopanic\n\t/usr/local/go1.19.1/src/runtime/panic.go:884\ngithub.com/pingcap/tidb/util/memory.(*PanicOnExceed).Action\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/action.go:146\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume.func1\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:404\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:440\ngithub.com/pingcap/tidb/executor.(*IndexLookUpJoin).newInnerWorker\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/index_lookup_join.go:242\ngithub.com/pingcap/tidb/executor.(*IndexLookUpJoin).startWorkers\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/index_lookup_join.go:194\ngithub.com/pingcap/tidb/executor.(*IndexLookUpJoin).Next\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/index_lookup_join.go:261\ngithub.com/pingcap/tidb/executor.Next\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/executor.go:324\ngithub.com/pingcap/tidb/executor.(*ExecStmt).next\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:990\ngithub.com/pingcap/tidb/executor.(*recordSet).Next\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/executor/adapter.go:153\ngithub.com/pingcap/tidb/server.(*tidbResultSet).Next\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/driver_tidb.go:407\ngithub.com/pingcap/tidb/server.(*clientConn).writeChunks\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2261\ngithub.com/pingcap/tidb/server.(*clientConn).writeResultset\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2204\ngithub.com/pingcap/tidb/server.(*clientConn).handleStmt\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:2071\ngithub.com/pingcap/tidb/server.(*clientConn).handleQuery\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1904\ngithub.com/pingcap/tidb/server.(*clientConn).dispatch\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1359\ngithub.com/pingcap/tidb/server.(*clientConn).Run\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/conn.go:1104\ngithub.com/pingcap/tidb/server.(*Server).onConn\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/server/server.go:563"]
[2022/10/18 18:44:31.530 +08:00] [ERROR] [coprocessor.go:808] ["copIteratorWork meet panic"] [r="\"Out Of Memory Quota![conn_id=6315680551852310935]\""] ["stack trace"="github.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTask.func1\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:810\nruntime.gopanic\n\t/usr/local/go1.19.1/src/runtime/panic.go:884\ngithub.com/pingcap/tidb/util/memory.(*PanicOnExceed).Action\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/action.go:146\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume.func1\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:404\ngithub.com/pingcap/tidb/util/memory.(*Tracker).Consume\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/util/memory/tracker.go:440\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).sendToRespCh\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:698\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopResponse\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1163\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleCopPagingResult\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:1000\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTaskOnce\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:939\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).handleTask\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:821\ngithub.com/pingcap/tidb/store/copr.(*copIteratorWorker).run\n\t/Users/pingcap/workspace/build-common/go/src/github.com/pingcap/tidb/store/copr/coprocessor.go:536"]
```
### 4. What is your TiDB version? (Required)
```sql
TiDB root@127.0.0.1:test> select * from information_schema.cluster_info;
+------+-----------------+-----------------+-------------+------------------------------------------+---------------------------+--------------+-----------+
| TYPE | INSTANCE | STATUS_ADDRESS | VERSION | GIT_HASH | START_TIME | UPTIME | SERVER_ID |
+------+-----------------+-----------------+-------------+------------------------------------------+---------------------------+--------------+-----------+
| tidb | 127.0.0.1:4000 | 127.0.0.1:10080 | 6.4.0-alpha | 17fac8bc2883dd287481a60f019beae948191a47 | 2022-10-18T18:43:43+08:00 | 3m54.283367s | 2872039 |
| pd | 127.0.0.1:2379 | 127.0.0.1:2379 | 6.4.0-alpha | bc2019e0c4a0eea1a9559f03a2f9de88511f793e | 2022-10-18T18:43:22+08:00 | 4m15.283374s | 0 |
| tikv | 127.0.0.1:20160 | 127.0.0.1:20180 | 6.4.0-alpha | 26b6c3cfccb386af50b1592ae4583dfb0003d7c1 | 2022-10-18T18:43:26+08:00 | 4m11.283376s | 0 |
+------+-----------------+-----------------+-------------+------------------------------------------+---------------------------+--------------+-----------+
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.