support stream encoding/decoding
- Dominant language
- Go
- Stars
- 95
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
We'd like to support OCF files, and potentially other kinds of streamed Avro record files.
Possible API:
```
// NewEncoder returns a new Encoder instance that encodes
// a stream of messages into r using the given schema.
// All messages encoded must have the same Avro schema.
//
// By default, it encodes the Avro object container file (OCF) format,
// writing the schema header when the first value is encoded.
func NewStreamEncoder(w io.Writer, schema string) *StreamEncoder
// Encode writes a message to the encoder.
// All messages must have the same Avro schema.
func (enc *StreamEncoder) Encode(x interface{}) error
// OmitHeader specifies that no file header should be written,
// just the encoded values themselves without the schema.
func (enc *StreamEncoder) OmitHeader()
type StreamDecoder struct {
readerSchema schema.AvroType
progs map[reflect.Type]*program
}
// NewStreamDecoder returns a new StreamDecoder instance that
// decodes a stream of messages from r.
// By default it decodes the Avro object container
// file (OCF) format, setting the schema by reading it
// from the start of the file.
// See https://avro.apache.org/docs/1.8.2/spec.html#Object+Container+Files
func NewStreamDecoder(r io.Reader) *StreamDecoder
// SetSchema sets the schema used to decode the
// message stream. If this is called before any messages
// are read, the decoder will not read the OCF
// header.
func (dec *StreamDecoder) SetSchema(schema string) error
// Buffered returns a reader of the data remaining in the Decoder's
// buffer. The reader is valid until the next call to Decode.
func (dec *StreamDecoder) Buffered() io.Reader
// Decode decodes the next value into x, which should
// be a pointer to an Avro-compatible type (see
// Unmarshal for further details).
func (dec *StreamDecoder) Decode(x interface{}) error
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.