bytedance / bytedance/bolt

[Performance] Aggregate functions: add DictionaryVector fast-path for numeric inputs

Open
#896 0 comments 0 reactions 1 assignee View on GitHub

@fzhedu is already working on this.

Since Aug 19, 2026.

performance
Dominant language
C++
Stars
177
Forks
107
Avg merge
3d 10h
Merged PRs (30d)
49

Description

Performance Bottleneck Type

CPU Bound (High CPU usage, slow calculation)

Environment Details (Critical)
  • CPU Model: generic x86_64 (any target)
  • SIMD Support: AVX2 / AVX-512
  • RAM: N/A (not memory bound)
  • Storage: N/A (in-memory Vector path)
  • OS & Kernel: Linux
  • Bolt Build Type: Release
Workload Description

Numeric aggregates (SUM / AVG / MIN / MAX / bitwise, etc.) whose input column
is a DictionaryVector, e.g. the output of a Filter, HashJoin build side, or
a low-cardinality column read from Parquet dict-encoded pages.

Query pattern:
  SELECT sum(v), avg(v), min(v), max(v)
  FROM t
  WHERE ...             -- produces dictionary-wrapped output
  GROUP BY k;

Data:  numeric column wrapped by DictionaryVector<int/float/double/...>.
Size:  applies to any scale; benefit grows with row count.
Performance Comparison
System Time / Latency Throughput (Rows/s) CPU Usage
Bolt (current, DecodedVector path) baseline baseline 100%
Bolt (proposed dictionary fast path) lower higher lower

The current base classes always call DecodedVector::decode() even when the
input is already a fully usable DictionaryVector, incurring an extra
allocation + populate step per batch.

Reproduction Steps
  1. Build Bolt in Release.
  2. Any TPC-H / TPC-DS query where a numeric column arrives at the aggregation
    as DictionaryVector (very common after a Filter or upstream dict scan).
  3. The new unit tests in AverageAggregationTest and SumTest
    (see PR) also exercise both addRawInput and addSingleGroupRawInput
    dictionary paths.
Profiling Data / Metrics

Root cause (from source inspection):

both fall through into a DecodedVector::decode(*arg, rows) path
regardless of the input encoding.

For DICTIONARY encoding we can iterate through valueAtFast(i) /
isNullAt(i) directly on the DictionaryVector and skip the
DecodedVector materialization entirely.

Additional context

Fix will be sent as a separate PR (linked to this issue). Guarded by
DictionaryVector::loadedVector() so it is safe when the dictionary values
underneath are still a lazy vector.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.