matrixorigin / matrixorigin/matrixone

[Bug] IVF PRE POST FILTER 测试: S3 LOAD DATA 10M 768维向量在 2CN 下 OOM 导致连接断开

Open
#27,137 1 comment 0 reactions 1 assignee Claimed by @slyunnn View on GitHub
kind/bug
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## 问题描述

Main TKE nightly regression 的 IVF PRE POST FILTER VECTOR RECALL TEST 在 Setup 阶段失败。S3 LOAD DATA 加载 10M 条 768 维向量数据,执行约 32 分钟后在 commit 时连接断开(error 2013)。

**2CN 环境下复现 OOM。**

## 错误

```
pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')
```

发生在 `conn.commit()` 阶段(S3 LOAD DATA 完成后提交时)。

## 时间线

```
22:49:27 开始 S3 LOAD DATA
bucket=mo-load-guangzhou-1308875761
filepath=vector/wiki_all_10m/historical_file_blocks_wiki_all_10m.csv
23:21:16 conn.commit() → Lost connection(约 32 分钟后)
```

## 环境

| Field | Value |
|-------|-------|
| 分支 | main |
| Commit | `ca73de9eec92c6c013a5f1a1533e179343e071b8` |
| 集群 | TKE(2CN + 1DN + 3Log + 1Proxy) |
| NSPrefix | mo-main |
| ClusterName | nightly-regression-dis |
| Run | https://github.com/matrixorigin/mo-nightly-regression/actions/runs/31714946606/job/94617295493 |

## 分析

10M 条 768 维向量(float32 = 3072 bytes/row,约 29GB 原始数据)的 S3 LOAD DATA 在 2CN 环境下执行时 CN OOM,导致连接断开。32 分钟的执行时间说明 LOAD DATA 过程中内存持续增长直至 OOM。

## 关联

- #26720 — IVF pre-filter 向量查询 EXPLAIN ANALYZE 导致 CN OOMKilled (10M 向量, 100 并发)
- #25142 — CN OOMKilled during IVF VECTOR INDEX TEST: INSERT...SELECT on vecf32(1024) table
- #24888 — CN OOM during IVF vector test data copy

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.