googleapis / googleapis/google-api-ruby-client
Streaming Google storage upload
- 主要言語
- Ruby
- スター
- 2.9k
- フォーク
- 888
- 平均マージ
- 6分
- マージ済み PR(30日)
- 218
説明
I am trying to solve the case of uploading files in a streaming fashion without keeping the full contents in memory
I have previously implemented the same feature in https://github.com/aws/aws-sdk-ruby/pull/1711
As identified in https://github.com/googleapis/google-cloud-ruby/issues/1997
There are some limitations in `google-api-client` which hard codes the use of the content length header
I have written a minimal wrapper library that monkey patches it here which should serve as a good source of inspiration: https://github.com/Fonsan/gcs_stream_upload
```ruby
require 'gcs_stream_upload'
storage = Google::Cloud::Storage.new(timeout: 5 * 60)
bucket = storage.bucket('bucket')
gcs_stream_upload = GCSStreamUpload.new(bucket)
gcs_stream_upload.upload('object') do |io|
IO.copy_stream(IO.popen('yes | head -n 10'), io)
end
# => Written "y\n" * 10
gcs_stream_upload.upload('object') do |io|
io << 'data'
io << 'dat2'
end
# => Written "datadat2"
```
コントリビューションガイド
調査の方向性
Read the limitations described in google-cloud-ruby issue #1997 and inspect the linked gcs_stream_upload wrapper, using its streaming examples as the starting point. Done means Google storage uploads can write streamed data without retaining the full contents in memory, including data produced through IO.copy_stream.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- ruby
- 領域
- cloud
- issue の種類
- 機能追加
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 25/100