googleapis / googleapis/google-cloud-go
bigquery/storage/managedwriter: better documentation for AppendRows usage across goroutines
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 1.6k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 109
Description
**Is your feature request related to a problem? Please describe.**
I'm trying to understand how to stream high-throughput data into the `default stream`. Should I be making a new `ManagedStream` for every single request and appending a single row? Should I instead make a single `ManagedStream` for the whole application instance and call `AppendRows` from each request? Is `AppendRows` safe for concurrent use? If it's not safe for concurrent use then should I have a single goroutine that bundles incoming rows and appends them in batches?
**Describe the solution you'd like**
There needs to be more docs on usage of the a `ManagedStream` in different cases and what you should and should not do. The existing [Best Practices](https://cloud.google.com/bigquery/docs/write-api-best-practices) docs do not cover the go client.
**Describe alternatives you've considered**
Alternatives would be just picking a path and seeing if it breaks but I'd be nice to get some guidance ahead of time.
**Additional context**
We have a centralized service that will be handling hundreds or thousands of rows per second and we believe the Storage Write API is the best choice for us. We plan to have a single "leader" instance accepting all rows to append from various sources but it's not obvious how the implementation should work to actually use the `ManagedWriter` at scale.
Contributor guide
Assessment
This issue has not been assessed yet.