drivendataorg / drivendataorg/cloudpathlib
MinIO custom S3 endpoint does not support our `touch` implementation
- Dominant language
- Python
- Stars
- 628
- Forks
- 88
- Avg merge
- 17h 28m
- Merged PRs (30d)
- 2
Description
Our touch implementation depends on the `MetadataDirective="REPLACE"`:
https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/cloudpathlib/s3/s3client.py#L158-L165
However, this fails our test for `touch` when the rig is `custom_s3_rig` running against our MinIO test server:
https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/tests/test_cloudpath_file_io.py#L61-L66
I suspect that MinIO doesn't support this S3 functionality yet or it is a bug. We potentially could workaround by checking if the `endpoint_url` is set to a non-AWS URL and potentially do the fallback entire copy in that case.
This needs research to understand if this is a MinIO bug we could file to be fixed upstream or if we should workaround it.
When this is fixed, we should re-enable this test.
Contributor guide
Assessment
This issue has not been assessed yet.