duckdb / duckdb/duckdb

Bug95: RemapList leaves list_entry_t array uninitialized, causing D_ASSERT crash in allocator

Open
#25,271 0 comments 0 reactions 0 assignees View on GitHub
reproduced
Dominant language
C++
Stars
41.2k
Forks
3.8k
Avg merge
2d 8h
Merged PRs (30d)
491

Description

### What happens?

In `src/function/scalar/struct/remap_struct.cpp`, `RemapList()` (lines 136–172) handles a `CONSTANT_VECTOR` list child by copying only `result[0]` via `memcpy`, while leaving `result[1..result_size-1]` as stale heap garbage — without ever calling `result.SetVectorType(VectorType::CONSTANT_VECTOR)` to signal that only index 0 is meaningful. When the enclosing struct is a `FLAT_VECTOR` (row-varying column reference) but the `LIST` sub-field is `CONSTANT_VECTOR` (pure literal expression), and a nested-remap mapping forces the recursive `RemapNested` → `RemapList` path, those garbage `list_entry_t{offset, length}` values propagate downstream. Downstream consumers use the garbage offset/length to size a memory allocation, tripping `D_ASSERT(size < MAXIMUM_ALLOC_SIZE)` in `Allocator::AllocateData` and crashing the process with an `INTERNAL Error`; in a release build (where `D_ASSERT` is compiled out) the same path produces an out-of-bounds heap read or OOM.

### To Reproduce

This is triggered entirely by SQL, with no external data files.

```sql
SELECT remap_struct(
{'id': i, 'lst': list_value(1, 2, 3)},
NULL::STRUCT(id INTEGER, lst2 INTEGER[]),
{'id': 'id', 'lst2': ROW('lst', {'list': 'list'})},
NULL
) FROM range(5) tbl(i);
```

```sh
# Run from the DuckDB repository root (the directory containing build/)
ASAN_OPTIONS=detect_leaks=0 ./build/debug/duckdb << 'EOF'
SELECT remap_struct(
{'id': i, 'lst': list_value(1, 2, 3)},
NULL::STRUCT(id INTEGER, lst2 INTEGER[]),
{'id': 'id', 'lst2': ROW('lst', {'list': 'list'})},
NULL
) FROM range(5) tbl(i);
EOF
```

```
INTERNAL Error: Assertion triggered in file "…/src/common/allocator/allocator.cpp" on line 122: false

Stack Trace:

/…/duckdb(+0x24c68a80) [0x61b0e7a43a80]
/…/duckdb(+0x24c68369) [0x61b0e7a43369]
/…/duckdb(+0x24c67cdb) [0x61b0e7a42cdb]
/…/duckdb(+0x24c72e50) [0x61b0e7a4de50]
/…/duckdb(+0x24e86385) [0x61b0e7c61385]
/…/duckdb(+0x24be1be7) [0x61b0e79bcbe7]
/…/duckdb(+0x2389c7cd) [0x61b0e66777cd]
/…/duckdb(+0x23b0b59f) [0x61b0e68e659f]
/…/duckdb(+0x23dc63ae) [0x61b0e6ba13ae]
/…/duckdb(+0x23d4dd4e) [0x61b0e6b28d4e]
/…/duckdb(+0x23d39eda) [0x61b0e6b14eda]

This error signals an assertion failure within DuckDB. This usually occurs due to unexpected conditions or errors in the program's logic.
For more information, see https://duckdb.org/docs/current/dev/internal_errors
```

### OS:

Ubuntu 24.04.2 LTS, x86_64 (Linux 6.8.0-55-generic)

### DuckDB Version:

v1.6.0-dev13633 (Development Version) 5b1ef771b2

### DuckDB Client:

CLI (Command Line Interface), built from source with `make debug` (ASan + UBSan instrumented)

### Hardware:

_No response_

### Full Name:

Yuelin

### Affiliation:

Individual

### Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

- [x] Yes, I have

### Did you include all code required to reproduce the issue?

- [x] Yes, I have

### Did you include all relevant data sets for reproducing the issue?

Yes

Contributor guide

Open the contributing guide

Research direction

Start in src/function/scalar/struct/remap_struct.cpp, especially RemapList() lines 136–172, and reproduce the failure with the provided SQL using a debug or ASan build. Trace the CONSTANT_VECTOR list-child path through RemapNested and verify that the query completes without the allocator assertion and returns valid list values for all rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.