golang / golang/go

proposal: encoding/json/jsontext: add Encoder.Flush method

Open
#75,475 0 comments 0 reactions 0 assignees View on GitHub
LibraryProposal Proposal
Dominant language
Go
Stars
139k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

### Proposal Details

I was trying to implement streaming Marshalling of JSON to send lazily produced values. Since the jsontext.Encoder buffers writes internally, there is no way to flush a completed value to the underlying writer.
By making the Flush Method accessible on the Encoder we could send a completed value.

What behavior did you expect:
Each completed Array Item is written to the underlying writer

What did you experience instead:
The first array Item is written to the stream and then nothing is written until all items are processed

Example Code:
```
// GOEXPERIMENT=jsonv2

package main

import (
"encoding/json/jsontext"
"encoding/json/v2"
"fmt"
"io"
"strconv"
"time"
)

type MyWork struct {
Items chan string
}

// No err checks for brevity
func main() {
r, w := io.Pipe()
original := MyWork{Items: make(chan string)}

go func() {
defer close(original.Items)
for i := range 5 {
<-time.After(1 * time.Second)
fmt.Printf("sending %d\n", i)
original.Items <- strconv.Itoa(i)
}
}()

go json.MarshalWrite(w, original, json.WithMarshalers(chanMarshaller))

target := MyWork{Items: make(chan string)}

go json.UnmarshalRead(r, &target, json.WithUnmarshalers(chanUnmarshaller))

for it := range target.Items {
fmt.Printf("Receiver got: %s\n", it)
}
}

var chanMarshaller = json.MarshalToFunc(
func(enc *jsontext.Encoder, items chan string) error {
enc.WriteToken(jsontext.BeginArray)

for it := range items {
json.MarshalEncode(enc, it)
// enc.Flush() could make sure the value is sent to the underlying writer
fmt.Printf("Marshaller written %s\n", it)
}
return enc.WriteToken(jsontext.EndArray)
})

var chanUnmarshaller = json.UnmarshalFromFunc(
func(dec *jsontext.Decoder, items *chan string) error {
defer close(*items)
dec.ReadToken()
for dec.PeekKind() != jsontext.EndArray.Kind() {
var it string
json.UnmarshalDecode(dec, &it)
*items <- it
}
dec.ReadToken()
return nil
})

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.