apache / apache/libcloud

Problem uploading empty files with upload_object

Ouverte
#1,562 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
2.1k
Forks
931
Merge moyen
1 j 2 h
PR mergées (30 j)
4

Description

## Summary

Using `upload_object` to upload an empty file returns response:
`400 - Bad Request: too many Host headers`

## Detailed Information

Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9

I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. `upload_object` function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
```
status: 400
reason: 'Bad Request: too many Host headers'
```

If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using `upload_object_via_stream` and found the following code works:
```
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
```
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).

I think using `upload_object` will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!

### Reproduce
1. `touch ~/test-empty.txt`
2. Use `upload_object` to upload test-empty.txt

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Les points d’entrée signalés sont upload_object et upload_object_via_stream ; commencez par reproduire le cas d’un fichier vide avec le pilote S3 et comparez le comportement entre AWS S3, MinIO et NetApp StorageGRID. Comme aucun fichier source ni aucun test n’est indiqué dans l’issue, localisez d’abord l’implémentation de l’upload et ses tests. C’est terminé lorsque upload_object gère les fichiers vides avec des métadonnées sans produire d’en-têtes Host en double ni compromettre la compatibilité avec les fournisseurs.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
aws, python
Domaine
cloud
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.