Skip to content

Parser: batch Vert.x eventbus messages to cut per-line envelope allocation #569

Description

@brunoborges

Summary

The parser ships one Vert.x eventbus message per parsed line. The per-message envelope objects (MessageImpl, DeliveryOptions, InboundDeliveryContext) add a measurable, constant allocation tax proportional to line count.

Evidence (JFR settings=profile, ~28s run)

Workload: sample Main parsing a 195 MB JDK8 ParallelGC rolling log (gclogs/rolling/jdk8/aagl_prd/gc.log), -Xmx1500m, G1, Java 25 (GraalVM 25.0.3). ~19,985 MB sampled allocations.

Allocation by class / site:

Item Share
io.vertx.core.eventbus.impl.MessageImpl 3.43% (~685 MB)
io.vertx.core.eventbus.DeliveryOptions 1.34%
io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext 1.30%
alloc site EventBusImpl.checkStarted() 3.94%

Combined ≈6% of sampled allocations is eventbus envelope overhead, one instance per line.

Suggested fixes

  • Batch multiple parsed lines/events into a single eventbus message to amortize the envelope cost (e.g. send chunks of N lines/events instead of one per line).
  • Consider a local (non-clustered) fast path for the single-process analysis case, avoiding full eventbus delivery machinery where a direct handler call would do.

Expected impact

Removes a large fraction of the ~6% eventbus-envelope allocation, and reduces checkStarted()/delivery-context CPU on every line. GC is already healthy (98.76% throughput); this is an allocation-throughput / scalability improvement, most visible on very large logs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions