googleapis / googleapis/google-api-ruby-client

Streaming Google storage upload

Abierto
#759 18 comentarios 2 reacciones 0 asignados Ver en GitHub
type: feature request
Lenguaje dominante
Ruby
Estrellas
2.9k
Forks
888
Merge medio
6 min
PR fusionados (30 d)
218

Descripción

I am trying to solve the case of uploading files in a streaming fashion without keeping the full contents in memory

I have previously implemented the same feature in https://github.com/aws/aws-sdk-ruby/pull/1711

As identified in https://github.com/googleapis/google-cloud-ruby/issues/1997
There are some limitations in `google-api-client` which hard codes the use of the content length header

I have written a minimal wrapper library that monkey patches it here which should serve as a good source of inspiration: https://github.com/Fonsan/gcs_stream_upload
```ruby
require 'gcs_stream_upload'

storage = Google::Cloud::Storage.new(timeout: 5 * 60)
bucket = storage.bucket('bucket')
gcs_stream_upload = GCSStreamUpload.new(bucket)

gcs_stream_upload.upload('object') do |io|
IO.copy_stream(IO.popen('yes | head -n 10'), io)
end
# => Written "y\n" * 10

gcs_stream_upload.upload('object') do |io|
io << 'data'
io << 'dat2'
end
# => Written "datadat2"
```

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Read the limitations described in google-cloud-ruby issue #1997 and inspect the linked gcs_stream_upload wrapper, using its streaming examples as the starting point. Done means Google storage uploads can write streamed data without retaining the full contents in memory, including data produced through IO.copy_stream.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
ruby
Área
cloud
Tipo de issue
Nueva funcionalidad
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.