Replace benchmark.cc with a Python script
Open
@liulx20 is already working on this.
Since Jun 5, 2026.
ci
engine
- Dominant language
- C++
- Stars
- 164
- Forks
- 33
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 96
Description
Background
bin/benchmark.cc directly uses google::protobuf::Arena to allocate QueryResponse messages. Because libneug.so hides all protobuf symbols via the version script (cmake/neug_exports.ld), the benchmark binary cannot link ${Protobuf_LIBRARIES} without causing duplicate libprotobuf runtime issues (#387).
PR #491 removed the explicit protobuf linkage from all other consumer targets, but benchmark.cc still has direct Arena usage that needs to be addressed.
Proposal
Replace bin/benchmark.cc with a Python script that uses neug_py_bind to run benchmarks. This:
- Eliminates the last direct protobuf dependency in consumer binaries
- Simplifies the build matrix (
benchmarktarget can be removed frombin/CMakeLists.txt) - Makes benchmarks easier to extend and parameterize without recompilation
- Leverages the existing Python binding infrastructure
Scope
- Port
benchmark.ccfunctionality to a Python script (config parsing, query execution loop, timing) - Remove
benchmark.ccand its CMake target frombin/CMakeLists.txt - Verify equivalent performance measurement accuracy through Python bindings
Related
- #387 — duplicate libprotobuf runtime bug
- #490 — remove explicit
${Protobuf_LIBRARIES}linkage - #491 — PR removing protobuf linkage from other targets
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.