[Bug] stream load 的时候报错 [INTERNAL_ERROR]unknown stream load id
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
doirs 版本 2.1.11
### What's Wrong?
错误日志:
I20260529 08:21:25.307652 198092 tablet.cpp:1980] cumulative compaction meet delete rowset, increase cumu point without other operation.|tablet id:=20711468|after cumulative compaction, cumu point:=2
I20260529 08:21:27.409749 199486 stream_load.cpp:208] new income streaming load request.id=ee4ad23d0593a0dd-b8e586767dbe51b2, job_id=-1, txn_id=-1, label=datax_doris_writer_55e3d16f-47bb-4282-9a82-603110fbbd10_0, elapse(s)=0, db=qxk, tbl=qxk_datahub_ysk_xj_gat_wlmqgj_wlmq_smzkxx_new, group_commit=0
I20260529 08:21:27.415176 199486 stream_load_executor.cpp:72] begin to execute stream load. label=datax_doris_writer_55e3d16f-47bb-4282-9a82-603110fbbd10_0, txn_id=11142908, query_id=ee4ad23d0593a0dd-b8e586767dbe51b2
I20260529 08:21:27.416150 199486 stream_load.cpp:214] finished to handle HTTP header, id=ee4ad23d0593a0dd-b8e586767dbe51b2, job_id=-1, txn_id=11142908, label=datax_doris_writer_55e3d16f-47bb-4282-9a82-603110fbbd10_0, elapse(s)=0
W20260529 08:21:27.417343 201197 status.h:415] meet error status: [INTERNAL_ERROR]unknown stream load id: ee4ad23d0593a0dd-b8e586767dbe51b2
...skipping one line
1# doris::vectorized::NewJsonReader::_open_file_reader(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# doris::vectorized::NewJsonReader::_simdjson_init_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::NewJsonReader::init_reader(std::unordered_map, std::allocator >, std::shared_ptr, std::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, std::shared_ptr > > > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4# doris::vectorized::VFileScanner::_get_next_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
6# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
7# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
8# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
9# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
10# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone
W20260529 08:21:27.417537 201197 status.h:415] meet error status: [INTERNAL_ERROR]failed to init reader, err: [INTERNAL_ERROR]unknown stream load id: ee4ad23d0593a0dd-b8e586767dbe51b2
0# doris::FileFactory::create_pipe_reader(doris::TUniqueId const&, std::shared_ptr*, doris::RuntimeState*, bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::NewJsonReader::_open_file_reader(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# doris::vectorized::NewJsonReader::_simdjson_init_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::NewJsonReader::init_reader(std::unordered_map, std::allocator >, std::shared_ptr, std::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, std::shared_ptr > > > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4# doris::vectorized::VFileScanner::_get_next_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
6# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
7# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
8# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
9# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
10# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone
0# doris::vectorized::VFileScanner::_get_next_reader() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
2# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
4# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
5# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
6# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
7# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
8# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
9# start_thread
10# __clone
W20260529 08:21:27.417586 201197 scanner_scheduler.cpp:283] Scan thread read VScanner failed: [INTERNAL_ERROR]failed to init reader, err: [INTERNAL_ERROR]unknown stream load id: ee4ad23d0593a0dd-b8e586767dbe51b2
0# doris::FileFactory::create_pipe_reader(doris::TUniqueId const&, std::shared_ptr*, doris::RuntimeState*, bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::NewJsonReader::_open_file_reader(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# doris::vectorized::NewJsonReader::_simdjson_init_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::NewJsonReader::init_reader(std::unordered_map, std::allocator >, std::shared_ptr, std::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, std::shared_ptr > > > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4# doris::vectorized::VFileScanner::_get_next_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
...skipping one line
7# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
8# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
9# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
10# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone
. cur path:
0# doris::vectorized::VFileScanner::_get_next_reader() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
2# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
4# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
5# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
6# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
7# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
8# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
9# start_thread
10# __clone
W20260529 08:21:27.417747 201068 task_scheduler.cpp:361] Pipeline task failed. query_id: ee4ad23d0593a0dd-b8e586767dbe51b2|ee4ad23d0593a0dd-b8e586767dbe51b3 reason: [INTERNAL_ERROR]failed to init reader, err: [INTERNAL_ERROR]unknown stream load id: ee4ad23d0593a0dd-b8e586767dbe51b2
0# doris::FileFactory::create_pipe_reader(doris::TUniqueId const&, std::shared_ptr*, doris::RuntimeState*, bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::NewJsonReader::_open_file_reader(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# doris::vectorized::NewJsonReader::_simdjson_init_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::NewJsonReader::init_reader(std::unordered_map, std::allocator >, std::shared_ptr, std::hash, std::allocator > >, std::equal_to, std::allocator > >, std::allocator, std::allocator > const, std::shared_ptr > > > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4# doris::vectorized::VFileScanner::_get_next_reader() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
6# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
7# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
8# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
9# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
10# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone
. cur path:
0# doris::vectorized::VFileScanner::_get_next_reader() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:489
2# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:0
4# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_release/doris/be/src/vec/exec/scan/vscanner.cpp:102
5# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
6# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
7# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
8# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
9# start_thread
10# __clone
I20260529 08:21:27.417892 201068 pipeline_fragment_context.cpp:175] PipelineFragmentContext::cancel|query_id=ee4ad23d0593a0dd-b8e586767dbe51b2|fragment_id=1|instance_id=ee4ad23d0593a0dd-b8e586767dbe51b3|reason=INTERNAL_ERROR|message=failed to init reader, err: [INTERNAL_ERROR]unknown stream load id: ee4ad23d0593a0dd-b8e586767dbe51b2
### What You Expected?
可以通过 stream load 加载数据
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start with be/src/runtime/stream_load/stream_load.cpp and stream_load_executor.cpp, then trace FileFactory::create_pipe_reader and NewJsonReader::_open_file_reader from the reported stack. Reproduce the stream load failure on Doris 2.1.11 and determine why the stream load ID is unavailable; done means the cause is explained and the request no longer reports the internal error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100