apache / apache/doris-flink-connector
[Bug] when use doris source to read data from doris-2.0,the be will crash
- Dominant language
- Java
- Stars
- 385
- Forks
- 279
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 6
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
apache-doris-2.0-beta-bin-x64-noavx2
flink-doris-connector-1.17.0
### What's Wrong?
when use DorisSourceFunction to query data:
`DataStreamSource> listDataStreamSource = env.addSource(new DorisSourceFunction(new DorisStreamOptions(props), new SimpleListDeserializationSchema()));`
then, be.out will print erro log:
I0808 09:57:44.648388 30942 fragment_mgr.cpp:745] Register query/load memory tracker, query/load id: bb343d2f65b046ae-a5e7eca8e02e1ede limit: 2.00 GB
I0808 09:57:44.648418 30942 plan_fragment_executor.cpp:115] PlanFragmentExecutor::prepare|query_id=TUniqueId(hi=-4957270016049264978, lo=-6491960127771500834)|instance_id=TUniqueId(hi=6071672337474381689, lo=-9070538737182937440)|backend_num=0|pthread_id=139938785482496
I0808 09:57:44.648770 7034 fragment_mgr.cpp:521] PlanFragmentExecutor::_exec_actual|query_id=bb343d2f65b046ae-a5e7eca8e02e1ede|instance_id=5442e9d263b9e779-821ef91388ca6ea0|pthread_id=139944866322176
I0808 09:57:44.648802 7034 plan_fragment_executor.cpp:253] PlanFragmentExecutor::open|query_id=TUniqueId(hi=-4957270016049264978, lo=-6491960127771500834)|instance_id=TUniqueId(hi=6071672337474381689, lo=-9070538737182937440)|mem_limit=2.00 GB
*** Query id: bb343d2f65b046ae-a5e7eca8e02e1ede ***
*** Aborted at 1691459864 (unix time) try "date -d @1691459864" if you are using GNU date ***
*** Current BE git commitID: afe6bb9 ***
*** SIGSEGV address not mapped to object (@0x40) received by PID 6627 (TID 7034 OR 0x7f47740b9700) from PID 64; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/src/doris/be/src/common/signal_handler.h:413
1# os::Linux::chained_handler(int, siginfo*, void*) in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
4# 0x00007F481A0D42F0 in /lib64/libc.so.6
5# memcpy at /root/src/doris/be/src/glibc-compatibility/memcpy/memcpy_x86_64.cpp:219
6# arrow::FixedSizeBinaryBuilder::AppendValues(unsigned char const*, long, unsigned char const*) in /home/data/apache-doris-2.0/be/lib/doris_be
7# doris::vectorized::DataTypeNumberSerDe<__int128>::write_column_to_arrow(doris::vectorized::IColumn const&, unsigned char const*, arrow::ArrayBuilder*, int, int) const at /root/src/doris/be/src/vec/data_types/serde/data_type_number_serde.cpp:86
8# doris::FromBlockConverter::convert(std::shared_ptr*) at /root/src/doris/be/src/util/arrow/block_convertor.cpp:392
9# doris::convert_to_arrow_batch(doris::vectorized::Block const&, std::shared_ptr const&, arrow::MemoryPool*, std::shared_ptr*) in /home/data/apache-doris-2.0/be/lib/doris_be
10# doris::vectorized::MemoryScratchSink::send(doris::RuntimeState*, doris::vectorized::Block*, bool) at /root/src/doris/be/src/vec/sink/vmemory_scratch_sink.cpp:83
11# doris::PlanFragmentExecutor::open_vectorized_internal() in /home/data/apache-doris-2.0/be/lib/doris_be
12# doris::PlanFragmentExecutor::open() at /root/src/doris/be/src/runtime/plan_fragment_executor.cpp:273
13# doris::FragmentExecState::execute() at /root/src/doris/be/src/runtime/fragment_mgr.cpp:263
14# doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /root/src/doris/be/src/runtime/fragment_mgr.cpp:527
15# std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
16# doris::ThreadPool::dispatch_thread() in /home/data/apache-doris-2.0/be/lib/doris_be
17# doris::Thread::supervise_thread(void*) at /root/src/doris/be/src/util/thread.cpp:466
18# start_thread in /lib64/libpthread.so.0
19# __clone in /lib64/libc.so.6
./bin/start_be.sh: 行 308: 6627 段错误 (吐核)${LIMIT:+${LIMIT}} "${DORIS_HOME}/lib/doris_be" "$@" 2>&1 < /dev/null
### What You Expected?
how to fix it?
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Reproduce the query through DorisSourceFunction with apache-doris-2.0-beta-bin-x64-noavx2 and flink-doris-connector-1.17.0, then trace the reported failure through src/vec/data_types/serde/data_type_number_serde.cpp and src/util/arrow/block_convertor.cpp. Confirm the crash is prevented for this input and add or update a regression test if the relevant test location is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100