googleapis / googleapis/google-api-ruby-client
Streaming Google storage upload
- Lenguaje dominante
- Ruby
- Estrellas
- 2.9k
- Forks
- 888
- Merge medio
- 6 min
- PR fusionados (30 d)
- 218
Descripción
I am trying to solve the case of uploading files in a streaming fashion without keeping the full contents in memory
I have previously implemented the same feature in https://github.com/aws/aws-sdk-ruby/pull/1711
As identified in https://github.com/googleapis/google-cloud-ruby/issues/1997
There are some limitations in `google-api-client` which hard codes the use of the content length header
I have written a minimal wrapper library that monkey patches it here which should serve as a good source of inspiration: https://github.com/Fonsan/gcs_stream_upload
```ruby
require 'gcs_stream_upload'
storage = Google::Cloud::Storage.new(timeout: 5 * 60)
bucket = storage.bucket('bucket')
gcs_stream_upload = GCSStreamUpload.new(bucket)
gcs_stream_upload.upload('object') do |io|
IO.copy_stream(IO.popen('yes | head -n 10'), io)
end
# => Written "y\n" * 10
gcs_stream_upload.upload('object') do |io|
io << 'data'
io << 'dat2'
end
# => Written "datadat2"
```
Guía de contribución
Línea de trabajo
Read the limitations described in google-cloud-ruby issue #1997 and inspect the linked gcs_stream_upload wrapper, using its streaming examples as the starting point. Done means Google storage uploads can write streamed data without retaining the full contents in memory, including data produced through IO.copy_stream.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- ruby
- Área
- cloud
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 25/100