pingcap / pingcap/tiflash

workload report “EstablishDisaggTask Failed4: Deadline Exceeded, e.what() = DB::Exception” after fault recover from s3 network_latency last for 10mins

Open
#7,889 1 comment 0 reactions 1 assignee View on GitHub

@JinheLin is already working on this.

Since Aug 2, 2023.

component/storage severity/moderate type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1、run workload
go-tpc ch run -D tpcc --host tc-tidb.endless-ha-test-tps-1830777-1-164 -P4000 --warehouses 2000 -T 32 --acThreads 1 --queries q22 --ignore-error '2013,1213,1105,1205,8022,8028,9004,9007,1062' --time 36000m --user root --password '' --interval '10s'
2、inject s3 network_latency(50ms) last for 10mins
inject fault:2023-08-01 19:39:00
recover fault:2023-08-01 19:49:00

2. What did you expect to see? (Required)

no error code from workload

3. What did you see instead (Required)

workload report error after fault recover from 2023-08-01 19:53:00
execute run failed, err execute query q22 failed Error 1105: other error for mpp stream: Code: 0, e.displayText() = DB::Exception: EstablishDisaggTask Failed4: Deadline Exceeded, e.what() = DB::Exception,\r\n

4. What is your TiFlash version? (Required)

githash:759bb3e19d8b381952ab644a78948d8cc0bf28c0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.