infiniflow / infiniflow/infinity

Embedded SDK silently truncates floats in mixed int/float arrays and sparse values

Open
#3,482 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
4.7k
Forks
445
Avg merge
2d 2h
Merged PRs (30d)
7

Description

### What happened?

In the embedded SDK, the constant-expression and `match_sparse` type dispatch looks at the first element only, so a mixed int/float collection is treated as all-integer and the fractional parts are lost:

```python
from infinity_embedded.local_infinity.utils import get_local_constant_expr_from_python_value

res = get_local_constant_expr_from_python_value([1, 2.5])
# res.literal_type == LiteralType.kIntegerArray
# 2.5 is handed to the engine's int64 array field - truncated or rejected

res = get_local_constant_expr_from_python_value({1: 5, 2: 0.5})
# res.literal_type == LiteralType.kLongSparseArray
# res.i64_array_value == [5, 0] <- int(v) in the SDK already truncated 0.5 to 0
```

`match_sparse` has the same first-element dispatch, so `table.match_sparse("c1", {1: 5, 2: 0.5}, "ip", 3)` and `SparseVector([1, 2], [5, 0.5])` lose the 0.5 the same way.

Reproduced on current `main` (`eca7266`).

### Expected behavior

Dispatch on all elements: an all-int collection stays an integer/long-sparse array; any float promotes the collection to a double (sparse) array with the values float-coerced. This is also what the HTTP SDK does implicitly: it sends the values as JSON and the server stores them as doubles, so today the embedded and HTTP SDKs silently disagree on the same insert.

### Versions

- infinity, current `main` (`eca7266`)
- `python/infinity_embedded/local_infinity/utils.py`, `python/infinity_embedded/local_infinity/query_builder.py`

Contributor guide

Open the contributing guide

Research direction

Start in python/infinity_embedded/local_infinity/utils.py at get_local_constant_expr_from_python_value, then inspect the match_sparse and SparseVector paths in python/infinity_embedded/local_infinity/query_builder.py. Check how mixed collections are classified and values are converted. Done means all-int inputs retain integer arrays, while any float promotes list and sparse values to double arrays without truncation, matching HTTP SDK behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.