apache / apache/libcloud

Problem uploading empty files with upload_object

Aperta
#1,562 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
2.1k
Fork
931
Merge medio
1g 2h
PR unite (30g)
4

Descrizione

## Summary

Using `upload_object` to upload an empty file returns response:
`400 - Bad Request: too many Host headers`

## Detailed Information

Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9

I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. `upload_object` function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
```
status: 400
reason: 'Bad Request: too many Host headers'
```

If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using `upload_object_via_stream` and found the following code works:
```
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
```
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).

I think using `upload_object` will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!

### Reproduce
1. `touch ~/test-empty.txt`
2. Use `upload_object` to upload test-empty.txt

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

I punti di ingresso segnalati sono upload_object e upload_object_via_stream; inizia riproducendo il caso di un file vuoto con il driver S3 e confrontando il comportamento tra AWS S3, MinIO e NetApp StorageGRID. Poiché nell’issue non sono indicati né un file sorgente né un test, individua prima l’implementazione dell’upload e i relativi test. Il lavoro è completato quando upload_object gestisce i file vuoti con metadati senza produrre header Host duplicati né compromettere la compatibilità con i provider.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, python
Ambito
cloud
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.