apache / apache/arrow-java

[Java] [Benchmarking] Large stdout when running benchmarks

Open
#378 3 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

Since ARROW-15058 running the Java benchmarks results in stdout logs that are extremely large (we're seeing them at ~14gb). This is disrupting running benchmarks on conbench.

We've temporarily worked around this by disabling stdout logging entirely, but this will make debugging issues in the future much harder.

Is there any way to reduce the volume of stdout logging when running these benchmarks?

cc @davisusanibar [~fan_li_ya]

**Reporter**: [Elena Henderson](https://issues.apache.org/jira/browse/ARROW-15259) / @ElenaHenderson

**Note**: *This issue was originally created as [ARROW-15259](https://issues.apache.org/jira/browse/ARROW-15259). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

The issue does not name a source file, benchmark entry point, or test. Start by reproducing a Java benchmark run and tracing the logging introduced since ARROW-15058; compare output volume with the temporary stdout-disabled workaround. Done means substantially smaller benchmark stdout while retaining useful debugging output and avoiding disruption to conbench.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.