apache / apache/beam

Restore proto round trip for Java DirectRunner (was: Performance degradation on the direct runner)

Open
#18,708 0 comments 0 reactions 0 assignees View on GitHub
bug direct P3 runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Running Nexmark queries with the direct runner between Beam 2.2.0 and 2.3.0 shows a performance degradation:

```

========================================
Beam 2.2.0 Beam 2.3.0
Query Runtime(sec)
Runtime(sec)
========================================
0000 6.4 10.6

0001 5.1 10.2
0002 3.0 5.8
0003 3.8
6.2
0004 0.9 1.4
0005 5.8 11.4
0006
0.8 1.4
0007 193.8 1249.1
0008 3.9 6.9

0009 0.9 1.3
0010 6.4 8.2
0011 5.0
9.4
0012 4.7 9.1

```

We can see especially Query 7 that is 10 times longer.

Imported from Jira [BEAM-3617](https://issues.apache.org/jira/browse/BEAM-3617). Original Jira may contain additional context.
Reported by: jbonofre.

Contributor guide

Open the contributing guide

Research direction

Start with the Nexmark queries on the Java DirectRunner and compare Beam 2.2.0 with 2.3.0 using the reported runtime table. Investigate the proto round-trip path indicated by the title, with Query 0007 as the main regression case; done means restoring the earlier performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.