matrixorigin / matrixorigin/matrixone
[Bug]: async IVFFLAT INCLUDE cannot maintain NaN or Infinity float values
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Environment
- MatrixOne commit: `01d60e1c4ded1b0f3fc1a4ecd75ce54e95e23b90`
- Deployment: local shared-TN cluster with two CNs
- Scope: asynchronous IVFFLAT INCLUDE maintenance through ISCP
## Problem
Asynchronous IVFFLAT maintenance cannot process an included `FLOAT` or `DOUBLE`
column containing `NaN`, `+Infinity`, or `-Infinity`. MatrixOne accepts and
stores these floating-point values in a base table, and a synchronous IVFFLAT
build copies them, but the async entries table remains empty or stops receiving
later DML.
Finite floating-point controls complete in the same cluster.
## Minimal reproduction
```sql
set experimental_ivf_index=1;
create database ivf_async_include_float_special;
use ivf_async_include_float_special;
create table t(
id bigint primary key,
f double,
v vecf32(2)
);
insert into t
select result,
cast('NaN' as double),
cast(concat('[',result,',0]') as vecf32(2))
from generate_series(1,10) g;
create index ix using ivfflat on t(v)
lists=1
op_type 'vector_l2_ops'
quantization 'float16'
include(f)
async;
```
After waiting well beyond the safe control's completion time:
```sql
select count(*) from ;
-- 0 (expected 10)
```
## Reproduction matrix
| INCLUDE type/value | async entries | synchronous entries |
|---|---:|---:|
| `DOUBLE NaN` | 0 / 10 | 10 / 10 |
| `DOUBLE +Infinity` | 0 / 10 | 10 / 10 |
| `DOUBLE -Infinity` | 0 / 10 | 10 / 10 |
| `FLOAT NaN` | 0 / 10 | 10 / 10 |
| finite `FLOAT/DOUBLE` | 10 / 10 | 10 / 10 |
Three independently created `DOUBLE NaN` async databases remained at zero
entries while finite controls reached their full row count.
## Root cause
`pkg/iscp/util.go:appendFloat64` emits finite values as numeric literals, but
emits the special values as bare tokens:
```go
if !math.IsInf(value, 0) {
strconv.AppendFloat(...) // emits NaN for NaN
} else if math.IsInf(value, 1) {
append("+Infinity")
} else {
append("-Infinity")
}
```
The generated ISCP `VALUES` SQL therefore contains `NaN`, `+Infinity`, or
`-Infinity` without a type-preserving cast or quoted literal. These tokens are
not reconstructed as the stored floating-point values when the batch SQL is
compiled.
## Expected behavior
Every valid `FLOAT` and `DOUBLE` value accepted by the base table, including
NaN and signed infinities, must round-trip through async IVFFLAT INCLUDE
maintenance without blocking the batch.
## Regression coverage
- `FLOAT` and `DOUBLE`: finite, NaN, +Infinity, -Infinity, +0, and -0;
- initial replay plus later INSERT/UPDATE;
- multiple special and finite values in one batch;
- synchronous build control;
- bit-level/sign-preserving comparison where applicable.
Contributor guide
Assessment
This issue has not been assessed yet.