matrixorigin / matrixorigin/matrixone

perf(hashmap): lazily allocate StrHash iterator scratch memory

Open
#27,234 1 comment 0 reactions 1 assignee Claimed by @Ariznawlll View on GitHub
phase/testing severity/s0
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Background

An earlier StrHash lazy-allocation prototype predates the current transactional iterator, owner-transfer, iterator-cache, exact allocation-accounting, and const-vector contracts. The old patch is not safe to transplant.

## Proposed change

- Lazily allocate fixed iterator scratch only on first non-empty use.
- Preserve transactional preview/commit generations across growth and invalidate them across owner changes or clear operations where required.
- Preserve exact MPool ownership for the key buffer while keeping Go-heap scratch outside MPool accounting.
- Include retained Go backing arrays in iterator-cache capacity decisions.
- Keep growth failure atomic and cleanup exactly once.

## Non-goals

- Cherry-picking the historical prototype.
- Adding locking to make one iterator concurrently usable.
- Introducing an active shrink policy in the same change.
- Combining the work with protobuf preallocation.
- Attributing the historical allocation-bundle result to this item.
- Any claim toward a 20% TPS improvement.

## Acceptance criteria

- Constructor, first use, reuse, growth/shrink sequence, owner transfer/clear, cache pruning, transactional preview/commit, failure injection, accounting, and direct HashBuild/Group consumers are covered.
- Focused race stress and owning-package race runs pass.
- Five-run benchmarks cover 1/2/8/16/256-row paths.
- Build, vet, relevant group/join smoke, semantic preflight, lifecycle self-review, and owner review gates pass before Ready.

## Historical evidence boundary

The earlier StrHash+Proto bundle measured `+1.4074%` to `+1.4817%` depending on baseline construction. It is directional bundle evidence only.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.