Problem uploading empty files with upload_object
- Lingua principale
- Python
- Stelle
- 2.1k
- Fork
- 931
- Merge medio
- 1g 2h
- PR unite (30g)
- 4
Descrizione
## Summary
Using `upload_object` to upload an empty file returns response:
`400 - Bad Request: too many Host headers`
## Detailed Information
Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9
I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. `upload_object` function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
```
status: 400
reason: 'Bad Request: too many Host headers'
```
If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using `upload_object_via_stream` and found the following code works:
```
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
```
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).
I think using `upload_object` will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!
### Reproduce
1. `touch ~/test-empty.txt`
2. Use `upload_object` to upload test-empty.txt
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
I punti di ingresso segnalati sono upload_object e upload_object_via_stream; inizia riproducendo il caso di un file vuoto con il driver S3 e confrontando il comportamento tra AWS S3, MinIO e NetApp StorageGRID. Poiché nell’issue non sono indicati né un file sorgente né un test, individua prima l’implementazione dell’upload e i relativi test. Il lavoro è completato quando upload_object gestisce i file vuoti con metadati senza produrre header Host duplicati né compromettere la compatibilità con i provider.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, python
- Ambito
- cloud
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100