apache / apache/beam

Golang Beam ParDo job not working as expected on Apple M1 chip

Open
#21,667 1 comment 0 reactions 0 assignees View on GitHub
bug go P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Reproducible example:

reproducible.go: [https://pastebin.com/raw/LqADath4](https://pastebin.com/raw/LqADath4)

go.mod: [https://pastebin.com/raw/qXmyh62N](https://pastebin.com/raw/qXmyh62N)

run cmd: go run reproducible.go \--output output.tsv

On lines 29-31, the program is supposed to read from rowIter function only while new data is available (this is standard interface to read input after GroupBy operation).

Instead, the cycle runs indefinitely. In gdb/debug prints it can be clearly seen that program just steps through the "if !b" block when "b" is false.

Supposedly this is the cause of UB from ABI function call inside beam.

 

Go version: go1.18 darwin/arm64

Imported from Jira [BEAM-14237](https://issues.apache.org/jira/browse/BEAM-14237). Original Jira may contain additional context.
Reported by: mgaiduk.

Contributor guide

Open the contributing guide

Research direction

Start with reproducible.go and go.mod from the linked examples, running `go run reproducible.go --output output.tsv` on Go 1.18 darwin/arm64. Inspect lines 29-31 and the rowIter handling after GroupBy; done means the ParDo job terminates when no new input is available instead of looping indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.