4paradigm / 4paradigm/OpenMLDB

SQL Engine assumes all the pointers are valid during the computation period while it is not always true

Open
#2,944 1 comment 0 reactions 2 assignees Claimed by @aceforeverd View on GitHub
bug high-priority
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

**Bug Description**
SQL Engine assumes all the pointers are valid during the computation period,
while it is not true in the storage engine.

For example,
for `*_cate` (see issue #2931), sql engine assumes that the `StringRef`s are valid during the whole computation, while in the `Iterator` implementation (see pr #2939), the `value` will become invalid/freed after the next call of `Next`.
- `FullTableIterator`: fixed in #2939, guarantee the `value` is valid in the whole life cycle of `Iterator`
- `RemoteWindowIterator`: as `sliding window` copies the rows, we didn't change the current behaviour for now
- For local `Iterator`, we haven't fixed it for now.

**Expected Behavior**
We have to work out a good solution between SQL engine and storage engine to make them have the same assumption.

**Relation Case**

**Steps to Reproduce**

1.
2.
3.
4.

Contributor guide

Open the contributing guide

Research direction

The issue references SQL engine and storage engine pointer validity, specifically StringRef in *_cate functions. Examine the Iterator implementations (FullTableIterator, RemoteWindowIterator, local Iterator) in the storage engine. Look at the SQL engine's assumption that pointers remain valid. Start by reading the code around issue #2931 and PR #2939 to understand the context. Determine what a 'good solution' for pointer lifetime coordination between the two subsystems would entail.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.