[Bug]: Calling a Java custom PTransform from Go using xlang fails on Dataflow Runner
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
As mentioned in #22931 - I am testing xlang support for the Go Runtime. I first was doing it wrong, using the Go direct runner, but then I moved on to test with the Dataflow runner.
I noticed that on Dataflow Runner, I could call a custom [Java Ptransform](https://github.com/ronoaldo/micro-beam/blob/main/05_xlang/java/src/main/java/com/ronoaldo/SplitWordsFromJava.java) from [Python](https://github.com/ronoaldo/micro-beam/blob/main/05_xlang/pipeline.py):

However,I could not call the same PTransform [from Go](https://github.com/ronoaldo/micro-beam/blob/main/05_xlang/go/pipeline.go). To submit the job, I started the Java expansion service as I did for the Python run, and called my Go pipeline. I see that the expansions service is called from Go, that the Go runtime stages the .jar files into Cloud Storage, but then the pipeline fails:

The only error message available is:
```
S03:e5/ProcessElementAndRestrictionWithSizing+External/FlatMapElements/FlatMap/ParMultiDo(Anonymous)+External/Filter/ParDo(Anonymous)/ParMultiDo(Anonymous)+stats.Count/stats.keyedCountFn+stats.Count/stats.SumPerKey/CombinePerKey/CoGBK+stats.Count/stats.SumPerKey/CombinePerKey/stats.sumIntFn/Partial+stats.Count/stats.SumPerKey/CombinePerKey/CoGBK/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers:
go-job-1-1662153285384063-09021416-tqko-harness-x8dv
Root cause: The worker lost contact with the service.,
go-job-1-1662153285384063-09021416-tqko-harness-x8dv
Root cause: The worker lost contact with the service.,
go-job-1-1662153285384063-09021416-tqko-harness-x8dv
Root cause: The worker lost contact with the service.,
go-job-1-1662153285384063-09021416-tqko-harness-x8dv
Root cause: The worker lost contact with the service.
```
Here is a full [pipeline log](https://docs.google.com/spreadsheets/d/1OFKBdDiHsz2ZAu5K_DjP4g15Wr2tr3rroyLN3qmts8o/edit?usp=sharing) and [worker logs](https://docs.google.com/spreadsheets/d/1aN8ycCBvs3tJljZ2gN2J0rv8GypQxebIcfM7tygGEeU/edit?usp=sharing) downloaded from Logs explorer.
Is this an actual bug or a unsupported workflow? I am trying to follow the Multi language pipelines section of the [Beam Programming Model docs](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines), regarding expose the Java PTransform (which I assume is correct since calling from Python works) and consuming it from Go, which I'm not sure if I missed any important steps.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-go
Contributor guide
Assessment
This issue has not been assessed yet.