apache / apache/fluss

Add fluss-microbench module: YAML-driven performance testing framework

Open
#3,080 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

There is currently no standardized way to run reproducible performance tests against Fluss. This makes it difficult to detect regressions, compare optimizations, and establish baselines — especially for aggregation merge engine workloads (RBM32/64, LISTAGG, SUM, etc.).

### Solution

Introduce a unified benchmarking module (fluss-microbench) that provides a CLI-driven performance testing framework for Fluss clusters.

Key Features:
- YAML-driven scenario configuration covering write, lookup, prefix-lookup, scan, and mixed workloads
- Dual-process architecture: Server (MiniCluster) runs in a forked JVM, Client runs in the main process, enabling independent resource sampling
- Five-layer metric collection: OS process (OSHI), JVM (MXBean), NMT, application-level (client + server metrics), and JFR
- Built-in presets: log-append, kv-upsert-get, kv-agg-mixed, kv-agg-listagg, kv-agg-rbm32, log-filter-pushdown
- Baseline management with --diff-previous and --diff-baseline for regression detection
- Structured JSON-lines stdout output, semantic exit codes, and --quick mode for fast iteration
- Dataset pre-generation for reproducible benchmarks
- HTML/CSV/JSON report generation with environment snapshots

CLI Commands:
- run, generate, validate, diff, baseline, list, clean

### Anything else?

_No response_

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by mapping the existing Fluss module structure and CLI conventions before designing the fluss-microbench module. Review the requested run, generate, validate, diff, baseline, list, and clean commands, then define how YAML scenarios, cluster processes, metrics, reports, and exit codes fit together. Done means the framework can run reproducible scenarios and produce the requested outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.