googleapis / googleapis/google-cloud-go
storage: Have Writer implement io.ReaderFrom
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 1.6k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 109
Description
Feature Request:
I have an `io.Reader` of data that I want to write to cloud storage using the google-cloud-go client library.
`b.bucket.Object(path).NewWriter(ctx)` returns an `io.Writer`
To copy the reader's data into the writer, I have to copy to an intermediate buffer, such as:
```go
writer := b.bucket.Object(path).NewWriter(ctx)
io.Copy(writer, reader)
```
(https://github.com/golang/go/blob/6cf6067d4eb20dfb3d31c0a8ccdbfdf0bf304b72/src/io/io.go#L387)
However, as an optimization, I would like to avoid this intermediary buffer and copy.
The `io.Copy` function checks if the writer implements `ReaderFrom`, and if it does, the intermediate allocation can be skipped.
See: https://github.com/golang/go/blob/6cf6067d4eb20dfb3d31c0a8ccdbfdf0bf304b72/src/io/io.go#L414
For all internally implemented `io.Reader`'s, I can also implement the `io.WriterTo` interface, to optimize this path. But, some of my reader's are from external sources that I cannot control.
This would be both a performance and UX improvement. AWS's S3 SDK and Azure's BlobStorage SDK for Go both use the pattern of accepting an `io.Reader` for writing data, rather than returning an `io.Writer` to use.
Contributor guide
Assessment
This issue has not been assessed yet.