pingcap / pingcap/tiflash

TSAN report data race on `DMFileReader::getReadRows()`

Open
#9,258 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/storage severity/moderate type/bug
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)
  1. Build gtests_dbms under tsan build type
  2. Run gtests_dbms --gtest_filter=*StoreIngestTest.ConcurrentIngestAndWrite*
2. What did you expect to see? (Required)
3. What did you see instead (Required)
==================
WARNING: ThreadSanitizer: data race (pid=2253513)
  Write of size 8 at 0x7b5c006600d0 by thread T118:
    #0 DB::DM::DMFileReader::getReadRows() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:155:10 (gtests_dbms+0xc2c5dd5)
    #1 DB::DM::DMFileReader::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:293:30 (gtests_dbms+0xc2c5dd5)
    #2 DB::DM::DMFileBlockInputStream::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.h:60:43 (gtests_dbms+0xc29b807)
    #3 DB::DM::ConcatSkippableBlockInputStream<false>::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SkippableBlockInputStream.cpp:131:34 (gtests_dbms+0xc21821e)
    #4 DB::DM::readBlock(std::__1::shared_ptr<DB::DM::SkippableBlockInputStream>&, std::__1::shared_ptr<DB::DM::SkippableBlockInputStream>&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadUtil.cpp:33:26 (gtests_dbms+0xc159f1f)
    #5 DB::DM::BitmapFilterBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/BitmapFilter/BitmapFilterBlockInputStream.cpp:40:32 (gtests_dbms+0xc23a213)
    #6 DB::DM::BitmapFilterBlockInputStream::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/BitmapFilter/BitmapFilterBlockInputStream.h:46:16 (gtests_dbms+0xc23ddf3)
    #7 DB::AddExtraTableIDColumnInputStream::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/DataStreams/AddExtraTableIDColumnInputStream.cpp:32:34 (gtests_dbms+0xd6d2e0e)
    #8 DB::DM::SegmentReadTaskPool::readOneBlock(std::__1::shared_ptr<DB::IBlockInputStream>&, std::__1::shared_ptr<DB::DM::SegmentReadTask> const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:238:26 (gtests_dbms+0xc2054a0)
    #9 DB::DM::MergedTask::readOneBlock() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:91:19 (gtests_dbms+0xc387656)
    #10 DB::DM::MergedTask::readBlock() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:32:12 (gtests_dbms+0xc387121)
    #11 DB::DM::SegmentReader::readSegments() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:67:39 (gtests_dbms+0xc3933b3)
    #12 DB::DM::SegmentReader::run() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:122:13 (gtests_dbms+0xc393127)
    #13 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xc394b83)
    #14 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xc394b83)
    #15 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xc394b83)

  Previous read of size 8 at 0x7b5c006600d0 by thread T58 (mutexes: write M739716239141892232):
    #0 DB::DM::DMFileReader::addCachedPacks(long, unsigned long, unsigned long, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&) const /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:644:9 (gtests_dbms+0xc2c9783)
    #1 DB::DM::DMFileReaderPoolSharding::set(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::DM::DMFileReader&, long, unsigned long, unsigned long, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/ColumnSharingCache.cpp:61:12 (gtests_dbms+0xc3745e3)
    #2 DB::DM::DMFileReaderPool::set(DB::DM::DMFileReader&, long, unsigned long, unsigned long, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/ColumnSharingCache.cpp:106:23 (gtests_dbms+0xc3745e3)
    #3 DB::DM::DMFileReader::readFromDiskOrSharingCache(DB::DM::ColumnDefine const&, COWPtr<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:634:38 (gtests_dbms+0xc2c8d62)
    #4 DB::DM::DMFileReader::readColumn(DB::DM::ColumnDefine const&, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:519:9 (gtests_dbms+0xc2c8391)
    #5 DB::DM::DMFileReader::readExtraColumn(DB::DM::ColumnDefine const&, unsigned long, unsigned long, unsigned long, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:485:23 (gtests_dbms+0xc2c76df)
    #6 DB::DM::DMFileReader::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:382:23 (gtests_dbms+0xc2c6ac8)
    #7 DB::DM::DMFileBlockInputStream::read() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.h:60:43 (gtests_dbms+0xc29b807)
    #8 DB::DM::ColumnFileBigReader::initStream() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ColumnFile/ColumnFileBig.cpp:196:38 (gtests_dbms+0xc2f711c)
    #9 DB::DM::ColumnFileBigReader::readRows(std::__1::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ColumnFile/ColumnFileBig.cpp:339:5 (gtests_dbms+0xc2f9570)
    #10 DB::DM::ColumnFileSetReader::readRows(std::__1::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long, unsigned long, DB::DM::RowKeyRange const*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> >*) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ColumnFile/ColumnFileSetReader.cpp:195:35 (gtests_dbms+0xc30956e)
    #11 DB::DM::ColumnFileSetReader::readPKVersion(unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ColumnFile/ColumnFileSetReader.cpp:135:5 (gtests_dbms+0xc308cba)
    #12 DB::DM::ColumnFileSetReader::getPlaceItems(std::__1::vector<DB::DM::BlockOrDelete, std::__1::allocator<DB::DM::BlockOrDelete> >&, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ColumnFile/ColumnFileSetReader.cpp:259:30 (gtests_dbms+0xc30a411)
    #13 DB::DM::DeltaValueReader::getPlaceItems(unsigned long, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Delta/Snapshot.cpp:200:29 (gtests_dbms+0xc36c023)
    #14 DB::DM::Segment::ensurePlace(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::shared_ptr<DB::DM::DeltaValueReader> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, unsigned long) const /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2604:32 (gtests_dbms+0xc1a82f3)
    #15 DB::DM::Segment::getReadInfo(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, DB::DM::ReadTag, unsigned long) const /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2436:44 (gtests_dbms+0xc18f943)
    #16 DB::DM::Segment::getInputStreamModeNormal(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:897:22 (gtests_dbms+0xc18c4b1)
    #17 DB::DM::Segment::buildBitmapFilterNormal(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2873:19 (gtests_dbms+0xc1acdd1)
    #18 DB::DM::Segment::buildBitmapFilter(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:2856:16 (gtests_dbms+0xc1a9b53)
    #19 DB::DM::Segment::getBitmapFilterInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:3310:26 (gtests_dbms+0xc18ebc4)
    #20 DB::DM::Segment::getInputStream(DB::DM::ReadMode const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:861:16 (gtests_dbms+0xc18bfea)
    #21 DB::DM::SegmentReadTask::doInitInputStream(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:313:29 (gtests_dbms+0xc1e767e)
    #22 DB::DM::SegmentReadTask::doInitInputStreamWithErrorFallback(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:282:9 (gtests_dbms+0xc1e7350)
    #23 DB::DM::SegmentReadTask::initInputStream(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:251:9 (gtests_dbms+0xc1e7169)
    #24 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:97:8 (gtests_dbms+0xc203a95)
    #25 DB::DM::MergedTask::initOnce() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:54:24 (gtests_dbms+0xc387292)
    #26 DB::DM::MergedTask::readBlock() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xc387119)
    #27 DB::DM::SegmentReader::readSegments() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:67:39 (gtests_dbms+0xc3933b3)
    #28 DB::DM::SegmentReader::run() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:122:13 (gtests_dbms+0xc393127)
    #29 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xc394b83)
    #30 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xc394b83)
    #31 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xc394b83)

  Location is heap block of size 784 at 0x7b5c0065fe00 allocated by thread T118:
    #0 operator new(unsigned long) /root/llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64:3 (gtests_dbms+0x2c757a8)
    #1 void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/new:235:10 (gtests_dbms+0xc2964e3)
    #2 std::__1::__libcpp_allocate(unsigned long, unsigned long) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/new:261:10 (gtests_dbms+0xc2964e3)
    #3 std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > >::allocate(unsigned long) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/allocator.h:108:38 (gtests_dbms+0xc2964e3)
    #4 std::__1::allocator_traits<std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > > >::allocate(std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > >&, unsigned long) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/allocator_traits.h:262:20 (gtests_dbms+0xc2964e3)
    #5 std::__1::__allocation_guard<std::__1::allocator<std::__1::__shared_ptr_emplace<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream> > > >::__allocation_guard<std::__1::allocator<DB::DM::DMFileBlockInputStream> >(std::__1::allocator<DB::DM::DMFileBlockInputStream>, unsigned long) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/allocation_guard.h:57:18 (gtests_dbms+0xc2964e3)
    #6 std::__1::shared_ptr<DB::DM::DMFileBlockInputStream> std::__1::allocate_shared<DB::DM::DMFileBlockInputStream, std::__1::allocator<DB::DM::DMFileBlockInputStream>, DB::DM::DMFileReader, bool, void>(std::__1::allocator<DB::DM::DMFileBlockInputStream> const&, DB::DM::DMFileReader&&, bool&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/shared_ptr.h:1105:48 (gtests_dbms+0xc2964e3)
    #7 std::__1::shared_ptr<DB::DM::DMFileBlockInputStream> std::__1::make_shared<DB::DM::DMFileBlockInputStream, DB::DM::DMFileReader, bool, void>(DB::DM::DMFileReader&&, bool&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/shared_ptr.h:1115:12 (gtests_dbms+0xc2964e3)
    #8 DB::DM::DMFileBlockInputStreamBuilder::build(std::__1::shared_ptr<DB::DM::DMFile> const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::ScanContext> const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileBlockInputStream.cpp:96:12 (gtests_dbms+0xc2964e3)
    #9 DB::DM::StableValueSpace::Snapshot::getInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long, bool, DB::DM::ReadTag, bool, bool, std::__1::vector<std::__1::shared_ptr<std::__1::set<unsigned long, std::__1::less<unsigned long>, std::__1::allocator<unsigned long> > >, std::__1::allocator<std::__1::shared_ptr<std::__1::set<unsigned long, std::__1::less<unsigned long>, std::__1::allocator<unsigned long> > > > > const&, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/StableValueSpace.cpp:549:35 (gtests_dbms+0xc2216f3)
    #10 DB::DM::Segment::getBitmapFilterInputStream(std::__1::shared_ptr<DB::DM::BitmapFilter>&&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:3110:72 (gtests_dbms+0xc1ad546)
    #11 DB::DM::Segment::getBitmapFilterInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:3338:12 (gtests_dbms+0xc18ee7a)
    #12 DB::DM::Segment::getInputStream(DB::DM::ReadMode const&, DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, unsigned long, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:861:16 (gtests_dbms+0xc18bfea)
    #13 DB::DM::SegmentReadTask::doInitInputStream(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:313:29 (gtests_dbms+0xc1e767e)
    #14 DB::DM::SegmentReadTask::doInitInputStreamWithErrorFallback(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:282:9 (gtests_dbms+0xc1e7350)
    #15 DB::DM::SegmentReadTask::initInputStream(std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, unsigned long, std::__1::shared_ptr<DB::DM::PushDownFilter> const&, DB::DM::ReadMode, unsigned long, bool) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTask.cpp:251:9 (gtests_dbms+0xc1e7169)
    #16 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:97:8 (gtests_dbms+0xc203a95)
    #17 DB::DM::MergedTask::initOnce() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:54:24 (gtests_dbms+0xc387292)
    #18 DB::DM::MergedTask::readBlock() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xc387119)
    #19 DB::DM::SegmentReader::readSegments() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:67:39 (gtests_dbms+0xc3933b3)
    #20 DB::DM::SegmentReader::run() /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:122:13 (gtests_dbms+0xc393127)
    #21 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xc394b83)
    #22 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xc394b83)
    #23 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xc394b83)

  Mutex M739716239141892232 is already destroyed.

  Thread T118 'SegmentReader' (tid=2253658, running) created by main thread at:
    #0 pthread_create /root/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x2c00bcd)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xc393205)
    #2 std::__1::thread::thread<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xc393205)
    #3 DB::DM::SegmentReader::SegmentReader(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:37:13 (gtests_dbms+0xc393008)
    #4 std::__1::__unique_if<DB::DM::SegmentReader>::__unique_single std::__1::make_unique<DB::DM::SegmentReader, DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&>(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xc38b883)
    #5 DB::DM::SegmentReaderPool::SegmentReaderPool(int, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:149:27 (gtests_dbms+0xc38b883)
    #6 std::__1::__unique_if<DB::DM::SegmentReaderPool>::__unique_single std::__1::make_unique<DB::DM::SegmentReaderPool, unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&>(unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xc38c398)
    #7 DB::DM::SegmentReaderPoolManager::init(unsigned int, double) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:190:32 (gtests_dbms+0xc38c398)
    #8 main /data/nvme0n1/jaysonhuang/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:75:50 (gtests_dbms+0x613b508)

  Thread T58 'SegmentReader' (tid=2253598, running) created by main thread at:
    #0 pthread_create /root/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x2c00bcd)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xc393205)
    #2 std::__1::thread::thread<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xc393205)
    #3 DB::DM::SegmentReader::SegmentReader(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:37:13 (gtests_dbms+0xc393008)
    #4 std::__1::__unique_if<DB::DM::SegmentReader>::__unique_single std::__1::make_unique<DB::DM::SegmentReader, DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&>(DB::DM::WorkQueue<std::__1::shared_ptr<DB::DM::MergedTask> >&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xc38b883)
    #5 DB::DM::SegmentReaderPool::SegmentReaderPool(int, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:149:27 (gtests_dbms+0xc38b883)
    #6 std::__1::__unique_if<DB::DM::SegmentReaderPool>::__unique_single std::__1::make_unique<DB::DM::SegmentReaderPool, unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&>(unsigned int&, std::__1::vector<int, std::__1::allocator<int> > const&) /data/nvme0n1/ra_common/tiflash-env-13/sysroot/bin/../include/c++/v1/__memory/unique_ptr.h:728:32 (gtests_dbms+0xc38c398)
    #7 DB::DM::SegmentReaderPoolManager::init(unsigned int, double) /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:190:32 (gtests_dbms+0xc38c398)
    #8 main /data/nvme0n1/jaysonhuang/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:75:50 (gtests_dbms+0x613b508)

SUMMARY: ThreadSanitizer: data race /data/nvme0n1/jaysonhuang/tiflash/dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:155:10 in DB::DM::DMFileReader::getReadRows()
==================
4. What is your TiFlash version? (Required)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce with a TSAN build using gtests_dbms --gtest_filter=StoreIngestTest.ConcurrentIngestAndWrite. Start in dbms/src/Storages/DeltaMerge/File/DMFileReader.cpp at getReadRows(), addCachedPacks(), and read(), then inspect ColumnSharingCache.cpp where the stack trace shows the concurrent access. Done means the test no longer reports a data race.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.