alibaba / alibaba/neug

Replace benchmark.cc with a Python script

Open
#495 1 comment 0 reactions 1 assignee View on GitHub

@liulx20 is already working on this.

Since Jun 5, 2026.

ci engine
Dominant language
C++
Stars
164
Forks
33
Avg merge
1d 9h
Merged PRs (30d)
96

Description

Background

bin/benchmark.cc directly uses google::protobuf::Arena to allocate QueryResponse messages. Because libneug.so hides all protobuf symbols via the version script (cmake/neug_exports.ld), the benchmark binary cannot link ${Protobuf_LIBRARIES} without causing duplicate libprotobuf runtime issues (#387).

PR #491 removed the explicit protobuf linkage from all other consumer targets, but benchmark.cc still has direct Arena usage that needs to be addressed.

Proposal

Replace bin/benchmark.cc with a Python script that uses neug_py_bind to run benchmarks. This:

  • Eliminates the last direct protobuf dependency in consumer binaries
  • Simplifies the build matrix (benchmark target can be removed from bin/CMakeLists.txt)
  • Makes benchmarks easier to extend and parameterize without recompilation
  • Leverages the existing Python binding infrastructure

Scope

  • Port benchmark.cc functionality to a Python script (config parsing, query execution loop, timing)
  • Remove benchmark.cc and its CMake target from bin/CMakeLists.txt
  • Verify equivalent performance measurement accuracy through Python bindings

Related

  • #387 — duplicate libprotobuf runtime bug
  • #490 — remove explicit ${Protobuf_LIBRARIES} linkage
  • #491 — PR removing protobuf linkage from other targets

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.