infiniflow / infiniflow/infinity

Thrift SDK silently truncates floats in mixed int/float arrays and sparse values

Open
#3,489 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
4.7k
Forks
445
Avg merge
2d 2h
Merged PRs (30d)
7

Description

### What happened?

In the thrift SDK (the default `infinity` package), the constant-expression and `match_sparse` type dispatch looks at the first element only, so a mixed int/float collection is treated as all-integer and the fractional parts are lost:

```python
from infinity.remote_thrift.utils import get_remote_constant_expr_from_python_value

res = get_remote_constant_expr_from_python_value([1, 2.5])
# res.literal_type == LiteralType.IntegerArray
# 2.5 is handed to the engine's int64 array field - truncated or rejected

res = get_remote_constant_expr_from_python_value({1: 5, 2: 0.5})
# res.literal_type == LiteralType.SparseIntegerArray
# res.i64_array_value == [5, 0] <- int(v) in the SDK already truncated 0.5 to 0
```

This affects every numeric collection shape on both paths:

- insert: one-dimensional arrays, tensors (`[[1, 2.5]]` becomes an integer tensor), tensor arrays, sparse dicts, and `SparseVector`
- query: `match_sparse` with a dict or a `SparseVector` (built by `make_match_sparse_expr`, same dispatch)

The HTTP SDK sends the same values as JSON and the server stores them as doubles, so the thrift and HTTP SDKs silently disagree on the same insert. The embedded SDK had the same dispatch; it is being fixed separately for one-dimensional arrays and sparse values in #3486.

Reproduced on current `main` (`eca7266`).

### Expected behavior

Dispatch on all elements: an all-int collection keeps its integer type; any float promotes the collection to the double type with the values float-coerced; anything else still raises.

### Versions

- infinity, current `main` (`eca7266`)
- `python/infinity_sdk/infinity/remote_thrift/utils.py`, `python/infinity_sdk/infinity/remote_thrift/types.py`

Contributor guide

Open the contributing guide

Research direction

Start in python/infinity_sdk/infinity/remote_thrift/utils.py, especially get_remote_constant_expr_from_python_value and make_match_sparse_expr, then review the related types in python/infinity_sdk/infinity/remote_thrift/types.py. Check the insert and match_sparse collection shapes described in the issue; done means all-int collections retain integer types, collections containing floats promote to doubles without truncation, and unsupported values still raise.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.