apache / apache/beam

[Bug]: Loop last element when write to file using textio in Go SDK

Open
#21,998 8 comments 0 reactions 0 assignees View on GitHub
bug go io P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

I wrote a test that writing to text file from a PCollection using go SDK.
Expected:
- Write each element of PCollection only one time to file and finish the task

Actual result:
- Each element of PCollection write to file only one time EXCEPT last element.
- Last element was keep writing infinitely to file and file keeps grow bigger and bigger.

Code:

```go
package main

import (
"context"
"flag"
"fmt"

"github.com/apache/beam/sdks/v2/go/pkg/beam"
_ "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/local"
"github.com/apache/beam/sdks/v2/go/pkg/beam/io/textio"
"github.com/apache/beam/sdks/v2/go/pkg/beam/runners/direct"
)

func main() {
flag.Parse()
beam.Init()
// In order to start creating the pipeline for execution, a Pipeline object is needed.
p := beam.NewPipeline()
s := p.Root()

firstPCol := beam.Create(s, "first", "mid1", "mid2", "mid3", "last")

textio.Write(s, "/Users/haupc/project/abeam/output.txt", firstPCol)

if _, err := direct.Execute(context.Background(), p); err != nil {
fmt.Printf("Pipeline failed: %v", err)
}
}
// expected:
/*
first
mid1
mid2
mid3
last
*/

// got
/*
first
mid1
mid2
mid3
last
last
last
last
last
last
last
last
last
last
last
...keep writing `last` and never stop
*/
```

### Issue Priority

Priority: 0

### Issue Component

Component: io-common

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.