googleapis / googleapis/google-cloud-go

storage: support for batch operations

Open
#2,530 0 comments 8 reactions 1 assignee Assigned to @cpriti-os View on GitHub
api: storage type: feature request
Dominant language
Go
Stars
4.5k
Forks
1.6k
Avg merge
1d 13h
Merged PRs (30d)
109

Description

**Is your feature request related to a problem? Please describe.**
In my app, I've got one-to-many relations between objects stored in SQL and some one-to-one reference between objects stored in SQL and in GCS. Let's say, in SQL I've got galleries and image_meta and in GCS -- images (each image_meta refers to one image). The reason why not to use buckets or folders is that image service doesn't know anything about galleries and gallery service -- about image names and storage structure.

**Describe the solution you'd like**
You have a [Batch API](https://cloud.google.com/storage/docs/json_api/v1/how-tos/batch). So it would be great to have something like
```go
batch := client.NewBatch()
batch.Add(
object0.DeleteAction(),
func(deleteActionResult *storage.ActionResult) {/* ... */},
func(err error) {/* ... */},
)
batch.Add(
object1.UpdateAction(),
func(deleteActionResult *storage.ActionResult) {/* ... */},
func(err error) {/* ... */},
)
// ...
batch.Add(
objectN.UpdateAction(),
func(deleteActionResult *storage.ActionResult) {/* ... */},
func(err error) {/* ... */},
)
batch.Exec(storage.WithNoWait() /** , some other options */).Do(ctx)
```

**Describe alternatives you've considered**
Now I'm using request per action with [Ants thread pool](https://github.com/panjf2000/ants):
```go
wg.Add(len(objects))
for _, obj := range objects {
pool.Submit(func() {
obj.Delete(ctx),
wg.Done()
})
}
wg.Wait()
```

Of cause, there could be some workaround like folders or buckets.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.