The acl parameter of open() is ignored when uploading an object smaller than blocksize
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
What happened:
with s3.open(s3_uri, 'wb', acl='bucket-owner-full-control') as f:
f.write(b'x' * (5 * 2 ** 20 - 1))
does not obey the acl parameter (the resulting owner is incorrect), whereas
with s3.open(s3_uri, 'wb', acl='bucket-owner-full-control') as f:
f.write(b'x' * (5 * 2 ** 20))
does.
What you expected to happen:
For the acl parameter to work regardless of object size.
Minimal Complete Verifiable Example:
with s3.open(s3_uri, 'wb', acl='bucket-owner-full-control') as f:
f.write(b'x' * (5 * 2 ** 20 - 1))
Anything else we need to know?:
I tested this with s3fs 0.4.2 because I can't currently install 0.5.x (dependency issues), but I suspect it is broken even on current master, because it's probably this if-statement causing the use of the single-part code path that does not use self.acl the way the multi-part code path does.
Environment:
- Dask version: 2.11.0
- Python version: 3.6.10
- Operating System: Linux (on SageMaker)
- Install method (conda, pip, source): It's SageMaker (conda) with a bunch of packages installed on top with pip, I'm not sure what exactly gets in as dependencies (but one is explicit s3fs<0.5). If this turns out to matter, I'll look into getting a cleaner install, but it probably doesn't.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in s3fs/core.py at the if-statement around line 1415 and compare the single-part path around line 1535 with the multipart path around line 1422. Reproduce the reported behavior with objects just below and at 5 MiB, then verify that the acl parameter is honored for both paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100