Problem uploading empty files with upload_object
- 主要言語
- Python
- スター
- 2.1k
- フォーク
- 931
- 平均マージ
- 1日 2時間
- マージ済み PR(30日)
- 4
説明
## Summary
Using `upload_object` to upload an empty file returns response:
`400 - Bad Request: too many Host headers`
## Detailed Information
Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9
I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. `upload_object` function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
```
status: 400
reason: 'Bad Request: too many Host headers'
```
If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using `upload_object_via_stream` and found the following code works:
```
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
```
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).
I think using `upload_object` will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!
### Reproduce
1. `touch ~/test-empty.txt`
2. Use `upload_object` to upload test-empty.txt
コントリビューションガイド
調査の方向性
報告されているエントリポイントは upload_object と upload_object_via_stream です。まず、S3 ドライバーで空ファイルのケースを再現し、AWS S3、MinIO、NetApp StorageGRID 間で動作を比較してください。Issue にはソースファイルもテストも記載されていないため、最初にアップロードの実装とそのテストを見つけてください。完了条件は、upload_object がメタデータ付きの空ファイルを処理でき、重複した Host ヘッダーを生成せず、プロバイダー互換性を損なわないことです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- aws, python
- 領域
- cloud
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100