googleapis / googleapis/google-api-ruby-client

Streaming Google storage upload

Open
#759 18 comments 2 reactions 0 assignees View on GitHub
type: feature request
Dominant language
Ruby
Stars
2.9k
Forks
888
Avg merge
6m
Merged PRs (30d)
218

Description

I am trying to solve the case of uploading files in a streaming fashion without keeping the full contents in memory

I have previously implemented the same feature in https://github.com/aws/aws-sdk-ruby/pull/1711

As identified in https://github.com/googleapis/google-cloud-ruby/issues/1997
There are some limitations in `google-api-client` which hard codes the use of the content length header

I have written a minimal wrapper library that monkey patches it here which should serve as a good source of inspiration: https://github.com/Fonsan/gcs_stream_upload
```ruby
require 'gcs_stream_upload'

storage = Google::Cloud::Storage.new(timeout: 5 * 60)
bucket = storage.bucket('bucket')
gcs_stream_upload = GCSStreamUpload.new(bucket)

gcs_stream_upload.upload('object') do |io|
IO.copy_stream(IO.popen('yes | head -n 10'), io)
end
# => Written "y\n" * 10

gcs_stream_upload.upload('object') do |io|
io << 'data'
io << 'dat2'
end
# => Written "datadat2"
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.