drivendataorg / drivendataorg/cloudpathlib

MinIO custom S3 endpoint does not support our `touch` implementation

Open
#141 0 comments 0 reactions 0 assignees View on GitHub
bug help wanted S3
Dominant language
Python
Stars
628
Forks
88
Avg merge
17h 28m
Merged PRs (30d)
2

Description

Our touch implementation depends on the `MetadataDirective="REPLACE"`:
https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/cloudpathlib/s3/s3client.py#L158-L165

However, this fails our test for `touch` when the rig is `custom_s3_rig` running against our MinIO test server:
https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/tests/test_cloudpath_file_io.py#L61-L66

I suspect that MinIO doesn't support this S3 functionality yet or it is a bug. We potentially could workaround by checking if the `endpoint_url` is set to a non-AWS URL and potentially do the fallback entire copy in that case.

This needs research to understand if this is a MinIO bug we could file to be fixed upstream or if we should workaround it.

When this is fixed, we should re-enable this test.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.