Add support for faster AWS S3 "Directory Buckets" storage
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
The problem
If you use external storage in Nextcloud, you'll notice it's not the fastest. There's a lag when you navigate directories, file operations take a few moments.
Describe your solution
Less than a year ago, AWS introduced "S3 Express One Zone" storage or "Directory Buckets". In exchange for reduced availability and reduced redundancy, you get faster file operations and (possibly) lower costs.
Instead of being replicated across multiple zones, directory buckets are in a single zone. According to AWS, directory buckets are up to 10x faster than standard S3 buckets and may reduce costs by up to 50%.
However, note that "Directory Buckets" do not support versioning.
Code changes
I don't anticipate big changes for the current AWS S3 bucket module. The library used with AWS may need to be updated. "Directory Buckets" prefer "session-based" authentication to get the full lower-latency benefits. So, extra AWS calls may need to be added to create sessions.
I did try connecting a "Directory Bucket" to Nextcloud 29 thinking there was a chance it might "just work", but I had no luck. These buckets use a different naming scheme, and Nextcloud 29 is complaining about the "invalid" naming scheme.
Additional context
AWS "Directory Buckets" docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-overview.html
Copying objects looks pretty much the same: https://docs.amazonaws.cn/en_us/AmazonS3/latest/userguide/directory-buckets-objects-copy.html#directory-bucket-copy-sdks
More info in this blog: https://www.pulumi.com/blog/amazon-s3-express-one-zone/
Directory Buckets (Express) vs Standard S3 Buckets
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the existing AWS S3 bucket module and the linked AWS Directory Buckets documentation first; inspect how Nextcloud 29 validates bucket names and handles authentication. Done means Directory Buckets are supported with their naming scheme and required session-based calls, while accounting for their lack of versioning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, php
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100