Golang Beam ParDo job not working as expected on Apple M1 chip
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Reproducible example:
reproducible.go: [https://pastebin.com/raw/LqADath4](https://pastebin.com/raw/LqADath4)
go.mod: [https://pastebin.com/raw/qXmyh62N](https://pastebin.com/raw/qXmyh62N)
run cmd: go run reproducible.go \--output output.tsv
On lines 29-31, the program is supposed to read from rowIter function only while new data is available (this is standard interface to read input after GroupBy operation).
Instead, the cycle runs indefinitely. In gdb/debug prints it can be clearly seen that program just steps through the "if !b" block when "b" is false.
Supposedly this is the cause of UB from ABI function call inside beam.
Go version: go1.18 darwin/arm64
Imported from Jira [BEAM-14237](https://issues.apache.org/jira/browse/BEAM-14237). Original Jira may contain additional context.
Reported by: mgaiduk.
Contributor guide
Research direction
Start with reproducible.go and go.mod from the linked examples, running `go run reproducible.go --output output.tsv` on Go 1.18 darwin/arm64. Inspect lines 29-31 and the rowIter handling after GroupBy; done means the ParDo job terminates when no new input is available instead of looping indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100