microsoft / microsoft/gctoolkit
Parser: batch Vert.x eventbus messages to cut per-line envelope allocation
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.3k
- Forks
- 177
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 3
Description
Summary
The parser ships one Vert.x eventbus message per parsed line. The per-message envelope objects (MessageImpl, DeliveryOptions, InboundDeliveryContext) add a measurable, constant allocation tax proportional to line count.
Evidence (JFR settings=profile, ~28s run)
Workload: sample Main parsing a 195 MB JDK8 ParallelGC rolling log (gclogs/rolling/jdk8/aagl_prd/gc.log), -Xmx1500m, G1, Java 25 (GraalVM 25.0.3). ~19,985 MB sampled allocations.
Allocation by class / site:
| Item | Share |
|---|---|
io.vertx.core.eventbus.impl.MessageImpl |
3.43% (~685 MB) |
io.vertx.core.eventbus.DeliveryOptions |
1.34% |
io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext |
1.30% |
alloc site EventBusImpl.checkStarted() |
3.94% |
Combined ≈6% of sampled allocations is eventbus envelope overhead, one instance per line.
Suggested fixes
- Batch multiple parsed lines/events into a single eventbus message to amortize the envelope cost (e.g. send chunks of N lines/events instead of one per line).
- Consider a local (non-clustered) fast path for the single-process analysis case, avoiding full eventbus delivery machinery where a direct handler call would do.
Expected impact
Removes a large fraction of the ~6% eventbus-envelope allocation, and reduces checkStarted()/delivery-context CPU on every line. GC is already healthy (98.76% throughput); this is an allocation-throughput / scalability improvement, most visible on very large logs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the parser's one-message-per-line path and the EventBusImpl.checkStarted() allocation site described in the issue. Use the supplied 195 MB JDK8 rolling-log workload with JFR profiling to compare batching or a local fast path; done means preserving parsing behavior while reducing the reported eventbus-envelope allocation and delivery overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100