Allow disabling of bucket validation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
In our organization we use S3 bucket names with : (such as s3://domain:bucket/path/file.txt) which come from our Ceph system. If we try to use s3fs with such a bucket name we get an error:
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid bucket name "rubin:rubinobs-lfa-cp": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"
with botocore we disable validation using:
>>> from botocore.handlers import validate_bucket_name
>>> client.meta.events.unregister("before-parameter-build.s3", validate_bucket_name)
And this works if I use s3._s3 to access the S3FileSystem object's internal boto client object.
Is there a way to do this without having to use an internal property?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at S3FileSystem and trace how its internal _s3 botocore client is created, including where validate_bucket_name is registered. Done means a supported public option can disable bucket validation for colon-containing names while keeping current validation as the default; check the project's relevant tests for coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100