open-policy-agent / open-policy-agent/java-opa-sdk

Add JMH benchmark suite

Open
#53 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance
Dominant language
Java
Stars
16
Forks
24
Avg merge
8h 17m
Merged PRs (30d)
15

Description

The current CLI's --count flag with --metrics (cli/README.md) produces rough timings, but it's not a substitute for proper microbenchmarks. Add a JMH-based benchmark module to measure hot paths and catch regressions.

  • New :benchmarks Gradle module wired with the JMH plugin (excluded from the published artifacts).
  • Cover the evaluator hot path: bundle load, plan prepare, single eval, and steady-state eval throughput.
  • Compare JSON adapters (opa-gson vs opa-jackson) on the same inputs.
  • Measure builtin invocation overhead (opa-builtins) — at least one hash, one regex, and one HTTP-free string builtin.
  • Provide a small set of representative policies + inputs as fixtures so runs are reproducible.
  • Document how to run (./gradlew :benchmarks:jmh) and how to interpret results in the module README.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading cli/README.md and inspecting the existing Gradle module configuration and published-artifact setup. Define the new :benchmarks module around the requested evaluator, adapter, and builtin cases, using representative policy and input fixtures. Done means ./gradlew :benchmarks:jmh runs reproducibly and the module README explains execution and result interpretation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, performance, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.