googleapis / googleapis/google-cloud-go
spanner: reuse decoding buffers from row to row
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 1.6k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 109
Description
The following statement is from a discussion:
It is nice that spanner package has Decoder interface, but we don't use it because it is hard to reuse buffers (needed for conversion) from row to row.
We have https://pkg.go.dev/go.chromium.org/luci/resultdb/internal/spanutil#Buffer which we allocate before a loop-over-rows and use for many rows (usage). Example:
```go
var b spanutil.Buffer
err := spanutil.Query(ctx, st, func(row *spanner.Row) error {
var key testVariantKey
if err := b.FromSpanner(row, &key.testID, &key.variantHash); err != nil {
return err
}
tvs[key] = struct{}{}
return nil
})
```
Here is a better one: https://chromium.googlesource.com/infra/luci/luci-go/+/HEAD/resultdb/internal/testresults/query.go#159
We compress some fields with zstd encoder, and reuse decoding buffer from row to row.
Compression is implemented here: https://chromium.googlesource.com/infra/luci/luci-go/+/HEAD/resultdb/internal/spanutil/compression.go
Contributor guide
Assessment
This issue has not been assessed yet.