Problem uploading empty files with upload_object
- Langage dominant
- Python
- Étoiles
- 2.1k
- Forks
- 931
- Merge moyen
- 1 j 2 h
- PR mergées (30 j)
- 4
Description
## Summary
Using `upload_object` to upload an empty file returns response:
`400 - Bad Request: too many Host headers`
## Detailed Information
Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9
I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. `upload_object` function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
```
status: 400
reason: 'Bad Request: too many Host headers'
```
If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using `upload_object_via_stream` and found the following code works:
```
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
```
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).
I think using `upload_object` will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!
### Reproduce
1. `touch ~/test-empty.txt`
2. Use `upload_object` to upload test-empty.txt
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Les points d’entrée signalés sont upload_object et upload_object_via_stream ; commencez par reproduire le cas d’un fichier vide avec le pilote S3 et comparez le comportement entre AWS S3, MinIO et NetApp StorageGRID. Comme aucun fichier source ni aucun test n’est indiqué dans l’issue, localisez d’abord l’implémentation de l’upload et ses tests. C’est terminé lorsque upload_object gère les fichiers vides avec des métadonnées sans produire d’en-têtes Host en double ni compromettre la compatibilité avec les fournisseurs.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- aws, python
- Domaine
- cloud
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 35/100