matrixorigin / matrixorigin/matrixone

[Performance]: make FULLTEXT2 loaded UUID membership probes allocation-free and evaluate raw-16 zero-copy

Open
#27,800 0 comments 0 reactions 1 assignee Claimed by @VioletQwQ-0 View on GitHub
area/performance severity/s1
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Problem

PR #27248 removes boxing and re-encoding for loaded FULLTEXT2 primary keys whose docmap bytes already match the runtime-filter probe representation. UUID remains on the typed fallback because the docmap stores a 36-byte canonical string while `docfilter` hashes the raw 16-byte UUID.

The loaded UUID path currently converts docmap bytes to `string`, parses them into `types.Uuid`, passes through `any`, and then probes the runtime filter with the raw 16 bytes. This is a follow-up to #27247 and #27248; it must not expand the scope of #27248.

## Stage P0: allocation-free UUID parse without a format change

- Read canonical UUID bytes from the loaded docmap.
- Parse from `[]byte` directly instead of converting to `string`.
- Reuse query-owned `[16]byte` scratch for the runtime-filter probe.
- Remove loaded-path UUID boxing and type assertion.
- Keep build-side segments on the current typed fallback.
- Preserve malformed-input fail-closed behavior.

This stage must not change SQL, DDL, CDC, FST, index format, runtime-filter wire protocol, membership results, or score bits.

## Stage P1: evaluate raw-16 true zero-copy

True zero-copy would store UUID as raw 16 bytes in the docmap so the loaded probe can borrow the mapped bytes directly. Do not implement this stage unless P0 still leaves UUID parsing as a material workload hotspot.

A P1 design must cover:

- an explicit docmap/index-format version;
- old-segment compatibility or safe fallback;
- REINDEX/rebuild and migration behavior;
- CDC, MERGE, restore, and mixed old/new segment handling;
- runtime-filter hash-input compatibility.

Until those gates are satisfied, raw-16 persistence remains deferred.

## Correctness and performance gates

- Loaded UUID hit, miss, and mixed probes match the current path.
- Malformed and truncated canonical UUID content fails safely.
- Build-side behavior remains unchanged.
- Membership results and `math.Float32bits(score)` remain identical.
- P0 reaches zero steady-state allocations or improves loaded UUID `ns/probe` by at least 20%.
- Benchmark hit/miss/mixed probes with GOMAXPROCS 1 and 10.
- Advance to P1 only when a workload profile shows UUID membership parsing remains material after P0.

## Non-goals

- No UUID production change in PR #27248.
- No cross-query cache or universal primary-key format migration in P0.
- No automatic index rebuild or format flip without a separate compatibility design.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.