translation from `sum` ast node to tipb::expr has incorrect field type
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
Current transformation from sum aggregation ast node to tipb::expr has some incorrect behaviors.
- output type must be unsigned
- output value must not be null, however
sum(null)should be null
And the cause might lies in Aggregation::toTiPBExecutor in dbms/src/Debug/astToExecutor.cpp:
if (agg_sig == tipb::ExprType::Count || agg_sig == tipb::ExprType::Sum)
{
auto * ft = agg_func->mutable_field_type();
ft->set_tp(TiDB::TypeLongLong);
ft->set_flag(TiDB::ColumnFlagUnsigned | TiDB::ColumnFlagNotNull);
}
1. Minimal reproduce step (Required)
By writing a simple test like
TEST_F(JoinExecutorTestRunner, agg)
try
{
context.addMockTable("tmp", "t", {{"a", TiDB::TP::TypeLong}}, {toNullableVec<Int32>({1, {}})});
auto request = context.scan("tmp", "t")
.aggregation({Sum(col("a"))}, {col("a")})
.build(context);
{
ASSERT_COLUMNS_EQ_R(createColumns({toNullableVec<Int64>({1, {}}), toNullableVec<Int64>({1, {}})}), executeStreams(request));
}
}
And execute ./gtests_dbms --gtest_filter="JoinExecutorTestRunner.agg"
2. What did you expect to see? (Required)
Test passed.
3. What did you see instead (Required)
Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type
Stack trace:
0x110988410 StackTrace::StackTrace() [gtests_dbms+4494296080]
dbms/src/Common/StackTrace.cpp:23
0x11098844c StackTrace::StackTrace() [gtests_dbms+4494296140]
dbms/src/Common/StackTrace.cpp:22
0x104b788c8 DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) [gtests_dbms+4295002312]
dbms/src/Common/Exception.h:41
0x104b77a20 DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) [gtests_dbms+4294998560]
dbms/src/Common/Exception.h:43
0x1073fadcc DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)::operator()(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long) const [gtests_dbms+4337479116]
dbms/src/Functions/FunctionsConversion.h:2884
0x1073fabb4 decltype(std::__1::forward<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)&>(fp)(std::__1::forward<DB::Block&>(fp0), std::__1::forward<std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&>(fp0), std::__1::forward<unsigned long>(fp0))) std::__1::__invoke<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)&, DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long>(DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)&, DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long&&) [gtests_dbms+4337478580]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/type_traits:3694
0x1073fab24 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)&, DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long>(DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)&, DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long&&) [gtests_dbms+4337478436]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/__functional_base:348
0x1073faabc std::__1::__function::__alloc_func<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long), std::__1::allocator<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>, void (DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>::operator()(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long&&) [gtests_dbms+4337478332]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/functional:1558
0x1073f9650 std::__1::__function::__func<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long), std::__1::allocator<DB::FunctionCast::prepare(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const::'lambda1'(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>, void (DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>::operator()(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long&&) [gtests_dbms+4337473104]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/functional:1732
0x10739027c std::__1::__function::__value_func<void (DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>::operator()(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long&&) const [gtests_dbms+4337042044]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/functional:1885
0x10738f004 std::__1::function<void (DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long)>::operator()(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long) const [gtests_dbms+4337037316]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/functional:2560
0x10738e908 DB::ExecutableFunctionCast::executeImpl(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long) const [gtests_dbms+4337035528]
dbms/src/Functions/FunctionsConversion.h:2491
0x10f63fb58 DB::IExecutableFunction::execute(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long) const [gtests_dbms+4474075992]
dbms/src/Functions/IFunction.cpp:230
0x105c9c8ac DB::IFunctionBase::execute(DB::Block&, std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > const&, unsigned long) const [gtests_dbms+4312975532]
dbms/src/Functions/IFunction.h:98
0x10e5817e8 DB::ExpressionAction::execute(DB::Block&) const [gtests_dbms+4456519656]
dbms/src/Interpreters/ExpressionActions.cpp:336
0x10e585784 DB::ExpressionActions::execute(DB::Block&) const [gtests_dbms+4456535940]
dbms/src/Interpreters/ExpressionActions.cpp:662
0x10d319b54 DB::ExpressionBlockInputStream::readImpl() [gtests_dbms+4437220180]
dbms/src/DataStreams/ExpressionBlockInputStream.cpp:53
0x10d3665a4 DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) [gtests_dbms+4437534116]
dbms/src/DataStreams/IProfilingBlockInputStream.cpp:75
0x10d366410 DB::IProfilingBlockInputStream::read() [gtests_dbms+4437533712]
dbms/src/DataStreams/IProfilingBlockInputStream.cpp:43
0x10d319af4 DB::ExpressionBlockInputStream::readImpl() [gtests_dbms+4437220084]
dbms/src/DataStreams/ExpressionBlockInputStream.cpp:50
0x10d3665a4 DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool) [gtests_dbms+4437534116]
dbms/src/DataStreams/IProfilingBlockInputStream.cpp:75
0x10d366410 DB::IProfilingBlockInputStream::read() [gtests_dbms+4437533712]
dbms/src/DataStreams/IProfilingBlockInputStream.cpp:43
0x106a35318 DB::tests::(anonymous namespace)::readBlock(std::__1::shared_ptr<DB::IBlockInputStream>) [gtests_dbms+4327232280]
dbms/src/TestUtils/ExecutorTestUtils.cpp:111
0x106a351e4 DB::tests::ExecutorTest::executeStreams(std::__1::shared_ptr<tipb::DAGRequest> const&, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > > > >&, unsigned long) [gtests_dbms+4327231972]
dbms/src/TestUtils/ExecutorTestUtils.cpp:127
0x106a354c0 DB::tests::ExecutorTest::executeStreams(std::__1::shared_ptr<tipb::DAGRequest> const&, unsigned long) [gtests_dbms+4327232704]
dbms/src/TestUtils/ExecutorTestUtils.cpp:132
0x104e898a0 DB::tests::JoinExecutorTestRunner_agg_Test::TestBody() [gtests_dbms+4298217632]
dbms/src/Flash/tests/gtest_join_executor.cpp:370
0x10c719db8 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [gtests_dbms+4424637880]
contrib/googletest/googletest/src/gtest.cc:2401
0x10c6f27f8 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [gtests_dbms+4424476664]
contrib/googletest/googletest/src/gtest.cc:2437
0x10c6f2754 testing::Test::Run() [gtests_dbms+4424476500]
contrib/googletest/googletest/src/gtest.cc:2473
0x10c6f358c testing::TestInfo::Run() [gtests_dbms+4424480140]
contrib/googletest/googletest/src/gtest.cc:2655
0x10c6f4284 testing::TestCase::Run() [gtests_dbms+4424483460]
contrib/googletest/googletest/src/gtest.cc:2773
0x10c6fad3c testing::internal::UnitTestImpl::RunAllTests() [gtests_dbms+4424510780]
contrib/googletest/googletest/src/gtest.cc:4673
unknown file: Failure
C++ exception with description "DB::Exception" thrown in the test body.
[ FAILED ] JoinExecutorTestRunner.agg (921 ms)
[----------] 1 test from JoinExecutorTestRunner (921 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (924 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] JoinExecutorTestRunner.agg
1 FAILED TEST
4. What is your TiFlash version? (Required)
master
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in dbms/src/Debug/astToExecutor.cpp at Aggregation::toTiPBExecutor, then review the JoinExecutorTestRunner.agg case in dbms/src/Flash/tests/gtest_join_executor.cpp. Run ./gtests_dbms --gtest_filter="JoinExecutorTestRunner.agg" and verify SUM output uses the intended unsigned field type while preserving NULL for SUM(NULL) without conversion errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100