matrixorigin / matrixorigin/matrixone

[Bug]: async IVFFLAT INCLUDE cannot maintain NaN or Infinity float values

Open
#29,019 0 comments 0 reactions 1 assignee Claimed by @iamlinjunhong View on GitHub
kind/bug needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Environment

- MatrixOne commit: `01d60e1c4ded1b0f3fc1a4ecd75ce54e95e23b90`
- Deployment: local shared-TN cluster with two CNs
- Scope: asynchronous IVFFLAT INCLUDE maintenance through ISCP

## Problem

Asynchronous IVFFLAT maintenance cannot process an included `FLOAT` or `DOUBLE`
column containing `NaN`, `+Infinity`, or `-Infinity`. MatrixOne accepts and
stores these floating-point values in a base table, and a synchronous IVFFLAT
build copies them, but the async entries table remains empty or stops receiving
later DML.

Finite floating-point controls complete in the same cluster.

## Minimal reproduction

```sql
set experimental_ivf_index=1;
create database ivf_async_include_float_special;
use ivf_async_include_float_special;

create table t(
id bigint primary key,
f double,
v vecf32(2)
);

insert into t
select result,
cast('NaN' as double),
cast(concat('[',result,',0]') as vecf32(2))
from generate_series(1,10) g;

create index ix using ivfflat on t(v)
lists=1
op_type 'vector_l2_ops'
quantization 'float16'
include(f)
async;
```

After waiting well beyond the safe control's completion time:

```sql
select count(*) from ;
-- 0 (expected 10)
```

## Reproduction matrix

| INCLUDE type/value | async entries | synchronous entries |
|---|---:|---:|
| `DOUBLE NaN` | 0 / 10 | 10 / 10 |
| `DOUBLE +Infinity` | 0 / 10 | 10 / 10 |
| `DOUBLE -Infinity` | 0 / 10 | 10 / 10 |
| `FLOAT NaN` | 0 / 10 | 10 / 10 |
| finite `FLOAT/DOUBLE` | 10 / 10 | 10 / 10 |

Three independently created `DOUBLE NaN` async databases remained at zero
entries while finite controls reached their full row count.

## Root cause

`pkg/iscp/util.go:appendFloat64` emits finite values as numeric literals, but
emits the special values as bare tokens:

```go
if !math.IsInf(value, 0) {
strconv.AppendFloat(...) // emits NaN for NaN
} else if math.IsInf(value, 1) {
append("+Infinity")
} else {
append("-Infinity")
}
```

The generated ISCP `VALUES` SQL therefore contains `NaN`, `+Infinity`, or
`-Infinity` without a type-preserving cast or quoted literal. These tokens are
not reconstructed as the stored floating-point values when the batch SQL is
compiled.

## Expected behavior

Every valid `FLOAT` and `DOUBLE` value accepted by the base table, including
NaN and signed infinities, must round-trip through async IVFFLAT INCLUDE
maintenance without blocking the batch.

## Regression coverage

- `FLOAT` and `DOUBLE`: finite, NaN, +Infinity, -Infinity, +0, and -0;
- initial replay plus later INSERT/UPDATE;
- multiple special and finite values in one batch;
- synchronous build control;
- bit-level/sign-preserving comparison where applicable.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.