apache / apache/beam

[Bug]: Nexmark Python fails to fetch metrics on Flink and Spark runners

Open
#24,737 1 comment 0 reactions 0 assignees View on GitHub
bug flink P3 python spark
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

This works on DirectRunner and Dataflow...
```
./gradlew --continue -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx6g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses -Pnexmark.runner=:runners:direct-java "-Pnexmark.args=--runner=DirectRunner --generateEventFilePathPrefix=gs://temp-storage-for-perf-tests/nexmark/eventFiles/jenkins-beam_PostCommit_Python_Nexmark_Flink_PR-5/query0- --query=0 --numEvents=100000 --manageResources=false --monitorJobs=true" :sdks:java:testing:nexmark:run
./gradlew -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx6g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses "-Pnexmark.args= --monitor_jobs --big_query_table=nexmark --big_query_dataset=nexmark_PRs --project=apache-beam-testing --resource_name_mode=QUERY_RUNNER_AND_MODE --export_summary_to_big_query --temp_location=gs://temp-storage-for-perf-tests/nexmark --influx_database=beam_test_metrics --influx_host=http://10.128.0.96:8086/ --base_influx_measurement=nexmark_PRs --export_summary_to_influx_db --influx_retention_policy=forever --suite=SMOKE --stream_timeout=60 --runner=FlinkRunner --query=0 --num_events=100000 --input=gs://temp-storage-for-perf-tests/nexmark/eventFiles/jenkins-beam_PostCommit_Python_Nexmark_Flink_PR-5/query0-\*" :sdks:python:apache_beam:testing:benchmarks:nexmark:run
```

14:51:25 Traceback (most recent call last):
14:51:25 File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
14:51:25 return _run_code(code, main_globals, None,
14:51:25 File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
14:51:25 exec(code, run_globals)
14:51:25 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Nexmark_Flink_PR/src/sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py", line 528, in
14:51:25 launcher.run()
14:51:25 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Nexmark_Flink_PR/src/sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py", line 513, in run
14:51:25 self.run_query(
14:51:25 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Nexmark_Flink_PR/src/sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py", line 312, in run_query
14:51:25 self.publish_performance_influxdb(query_num, perf)
14:51:25 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Nexmark_Flink_PR/src/sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py", line 400, in publish_performance_influxdb
14:51:25 mt = ','.join([measurement] + [k + "=" + v for k, v in tags.items()])
14:51:25 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Nexmark_Flink_PR/src/sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py", line 400, in
14:51:25 mt = ','.join([measurement] + [k + "=" + v for k, v in tags.items()])
14:51:25 TypeError: can only concatenate str (not "NoneType") to str

### Issue Priority

Priority: 3 (minor)

### Issue Components

- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [X] Component: Spark Runner
- [X] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Research direction

Start with sdks/python/apache_beam/testing/benchmarks/nexmark/nexmark_launcher.py, especially run_query and publish_performance_influxdb around line 400, then reproduce the provided Nexmark command with FlinkRunner or a Spark runner. Trace which metric tag is None and verify that performance metrics can be published without the reported TypeError on Flink and Spark while the existing DirectRunner behavior remains working.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, spark
Domain
data-engineering, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.