The error and info log is printed in one line
Open
Nobody has claimed this yet.
component/storage
may-affects-4.0
may-affects-5.0
may-affects-5.1
may-affects-5.2
may-affects-5.3
may-affects-5.4
may-affects-6.1
may-affects-6.5
severity/minor
type/bug
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
Tiflash oom and the log is as follows. The error and info log is printed in one line and it troubles with the log search.
[2023/02/22 15:19:42.619 +08:00] [ERROR] [SegmentReader.cpp:123] ["ErrMsg: Memory limit (total) exceeded caused by 'RSS(Resident Set Size) much larger than limit' : process memory size would be 47.89 GiB for (attempt to allocate chunk of 1049120 bytes), limit of memory for data computing : 28.63 GiB StackTrace \n 0x1a03fb1\tDB::TiFlashException::TiFlashException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::TiFlashError const&) [tiflash+27279281]\n \tdbms/src/Common/TiFlashException.h:250\n 0x1a03543\tMemoryTracker::alloc(long, bool) [tiflash+27276611]\n \tdbms/src/Common/MemoryTracker.cpp:152\n 0x1a03205\tMemoryTracker::alloc(long, bool) [tiflash+27275781]\n \tdbms/src/Common/MemoryTracker.cpp:163\n 0x1a03205\tMemoryTracker::alloc(long, bool) [tiflash+27275781]\n \tdbms/src/Common/MemoryTracker.cpp:163\n 0x1a1326b\tAllocator<false>::alloc(unsigned long, unsigned long) [tiflash+27341419]\n \tdbms/src/Common/Allocator.cpp:68\n 0x72378f5\tDB::createReadBufferFromFileBaseByFileProvider(std::__1::shared_ptr<DB::FileProvider> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::EncryptionPath const&, unsigned long, std::__1::shared_ptr<DB::ReadLimiter> const&, DB::ChecksumAlgo, unsigned long, int) [tiflash+119765237]\n \tdbms/src/Encryption/createReadBufferFromFileBaseByFileProvider.cpp:86\n 0x1962848\tDB::CompressedReadBufferFromFileProvider<false>::CompressedReadBufferFromFileProvider(std::__1::shared_ptr<DB::FileProvider>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::EncryptionPath const&, unsigned long, std::__1::shared_ptr<DB::ReadLimiter> const&, DB::ChecksumAlgo, unsigned long) [tiflash+26617928]\n \tdbms/src/Encryption/CompressedReadBufferFromFileProvider.cpp:78\n 0x7a068d8\tDB::DM::DMFileReader::Stream::Stream(DB::DM::DMFileReader&, long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned long, unsigned long, std::__1::shared_ptr<DB::Logger> const&, std::__1::shared_ptr<DB::ReadLimiter> const&) [tiflash+127953112]\n \tdbms/src/Storages/DeltaMerge/File/DMFileReader.cpp:158\n 0x7a0ddd4\tstd::__1::__function::__func<DB::DM::DMFileReader::DMFileReader(std::__1::shared_ptr<DB::DM::DMFile> const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, bool, bool, bool, bool, unsigned long, DB::DM::DMFilePackFilter&&, std::__1::shared_ptr<DB::MarkCache> const&, bool, std::__1::shared_ptr<DB::DM::ColumnCache> const&, unsigned long, unsigned long, std::__1::shared_ptr<DB::FileProvider> const&, std::__1::shared_ptr<DB::ReadLimiter> const&, unsigned long, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::shared_ptr<DB::DM::ScanContext> const&)::$_2, std::__1::allocator<DB::DM::DMFileReader::DMFileReader(std::__1::shared_ptr<DB::DM::DMFile> const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, bool, bool, bool, bool, unsigned long, DB::DM::DMFilePackFilter&&, std::__1::shared_ptr<DB::MarkCache> const&, bool, std::__1::shared_ptr<DB::DM::ColumnCache> const&, unsigned long, unsigned long, std::__1::shared_ptr<DB::FileProvider> const&, std::__1::shared_ptr<DB::ReadLimiter> const&, unsigned long, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::shared_ptr<DB::DM::ScanContext> const&)::$_2>, void (std::__1::vector<DB::IDataType::Substream, std::__1::allocator<DB::IDataType::Substream> > const&)>::operator()(std::__1::vector<DB::IDataType::Substream, std::__1::allocator<DB::IDataType::Substream> > const&) [tiflash+127983060]\n \t/usr/local/bin/../include/c++/v1/__functional/function.h:345\n 0x7a077f0\tDB::DM::DMFileReader::DMF[2023/02/22 15:19:47.639 +08:00] [INFO] [BaseDaemon.cpp:1160] ["Welcome to TiFlash"] [thread_id=1]
[2023/02/22 15:19:47.639 +08:00] [INFO] [BaseDaemon.cpp:1161] ["Starting daemon with revision 54381"] [thread_id=1]
[2023/02/22 15:19:47.639 +08:00] [INFO] [BaseDaemon.cpp:1164] ["TiFlash build info: TiFlash\nRelease Version: v6.7.0-alpha\nEdition: Community\nGit Commit Hash: 2d5909edc07402992ba1ff7c81f7aafeb227ccd3\nGit Branch: heads/refs/tags/v6.7.0-alpha\nUTC Build Time: 2023-02-21 11:35:50\nEnable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto\nProfile: RELWITHDEBINFO\n"] [thread_id=1]

2. What did you expect to see? (Required)
3. What did you see instead (Required)
4. What is your TiFlash version? (Required)
master
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the log output around SegmentReader.cpp:123 and the startup entries in BaseDaemon.cpp:1160-1164. Reproduce the TiFlash OOM scenario and trace how the error and subsequent INFO records are emitted. Done means separate log records remain separately searchable while preserving the error stack trace.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100