new feature: allow users to inject customized HTTP header
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Hi team, there're a lot of storage backends which declare themself to be S3 compatible, but they have slight differences.
One concrete example is https://github.com/apache/opendal/issues/7821, netapp S3 compatible object storage requires content-length in copy request, otherwise the request will get rejected.
OpenDAL, from my perspective, should focus on the well-documented S3 spec, instead of adding undocumented logic for different vendor impls.
- Each vendors' implementation could conflict with each other
- Support different types of S3 backend in OpenDAL core increases maintenance overhead
+ For example, netapp S3 API is not open source, with undocumented request format, no docker image provided, I don't even know how can we test and maintain
So I'm wondering if it's better to allow users to inject their own header.
- User-side should have little overhead to maintain a small list of headers
- Vendors could have their customized headers for performance (the object storage I'm working does so)
This feature request is not specific to S3 backend, but should be general enough for all backends.
### Problem and Solution
For all the operator APIs, we allow users to inject their own headers, which will be propagated to HTTP requests.
### Additional Context
See original PR and thread for details at https://github.com/apache/opendal/pull/7884#issuecomment-4926332061
### Are you willing to contribute to the development of this feature?
- [x] Yes, I am willing to contribute to the development of this feature.
Contributor guide
Research direction
Start with the original PR #7884 and its linked discussion, then trace how the operator APIs construct HTTP requests across storage backends. Define how user-provided headers should be supplied and propagated generally, and consider how the behavior can be verified for all affected operator APIs without adding vendor-specific logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100