microsoft / microsoft/gctoolkit

Parser: batch Vert.x eventbus messages to cut per-line envelope allocation

Open
#569 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
1.3k
Forks
177
Avg merge
1d 11h
Merged PRs (30d)
3

Description

Summary

The parser ships one Vert.x eventbus message per parsed line. The per-message envelope objects (MessageImpl, DeliveryOptions, InboundDeliveryContext) add a measurable, constant allocation tax proportional to line count.

Evidence (JFR settings=profile, ~28s run)

Workload: sample Main parsing a 195 MB JDK8 ParallelGC rolling log (gclogs/rolling/jdk8/aagl_prd/gc.log), -Xmx1500m, G1, Java 25 (GraalVM 25.0.3). ~19,985 MB sampled allocations.

Allocation by class / site:

Item Share
io.vertx.core.eventbus.impl.MessageImpl 3.43% (~685 MB)
io.vertx.core.eventbus.DeliveryOptions 1.34%
io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext 1.30%
alloc site EventBusImpl.checkStarted() 3.94%

Combined ≈6% of sampled allocations is eventbus envelope overhead, one instance per line.

Suggested fixes

  • Batch multiple parsed lines/events into a single eventbus message to amortize the envelope cost (e.g. send chunks of N lines/events instead of one per line).
  • Consider a local (non-clustered) fast path for the single-process analysis case, avoiding full eventbus delivery machinery where a direct handler call would do.

Expected impact

Removes a large fraction of the ~6% eventbus-envelope allocation, and reduces checkStarted()/delivery-context CPU on every line. GC is already healthy (98.76% throughput); this is an allocation-throughput / scalability improvement, most visible on very large logs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the parser's one-message-per-line path and the EventBusImpl.checkStarted() allocation site described in the issue. Use the supplied 195 MB JDK8 rolling-log workload with JFR profiling to compare batching or a local fast path; done means preserving parsing behavior while reducing the reported eventbus-envelope allocation and delivery overhead.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.