Tracking Issue: Complete object_store Integration Implementation
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
## Overview
This issue tracks the remaining work needed to fully implement the `object_store` trait in OpenDAL's integration layer. While the core functionality is working, several methods return `NotSupported` errors and some implementations could be enhanced.
## Current Status
### ✅ Implemented
- `put_opts` with full PutMode support (Overwrite, Create, Update)
- `put_multipart` (basic implementation)
- `get_opts` with GetOptions support (range, conditionals, version)
- `delete`
- `list`, `list_with_offset`, `list_with_delimiter`
- `MultipartUpload` trait
- Error mapping between OpenDAL and object_store
- Amazon S3 builder support
### ❌ Not Implemented
The following methods currently return `NotSupported` errors:
1. **`put_multipart_opts`** ([store.rs:165](integrations/object_store/src/store.rs:165))
- Required for advanced multipart upload options
- Currently returns: "put_multipart_opts is not implemented so far"
2. **`copy`** ([store.rs:248](integrations/object_store/src/store.rs:248))
- Server-side copy operation
- Currently returns: "copy is not implemented so far"
3. **`rename`** ([store.rs:255](integrations/object_store/src/store.rs:255))
- Server-side rename operation
- Currently returns: "rename is not implemented so far"
4. **`copy_if_not_exists`** ([store.rs:262](integrations/object_store/src/store.rs:262))
- Conditional server-side copy
- Currently returns: "copy_if_not_exists is not implemented so far"
### ⚠️ Incomplete Implementations
1. **PutResult metadata**
- Always returns `e_tag: None, version: None`
- Should return actual ETag and version when available from backend
2. **GetResult attributes**
- Always returns `Default::default()` for attributes
- Should include object metadata/attributes from backend
3. **Test coverage**
- Missing behavior tests for list operations
- No tests for `list_with_delimiter` and `list_with_offset`
## Tasks
### High Priority
- [ ] Implement `copy` method using OpenDAL's copy operation
- [ ] Implement `rename` method using OpenDAL's rename operation
- [ ] Return proper ETag in `PutResult` when available
- [ ] Return proper version in `PutResult` when backend supports versioning
### Medium Priority
- [ ] Implement `put_multipart_opts` with proper options handling
- [ ] Implement `copy_if_not_exists` with conditional logic
- [ ] Populate `GetResult.attributes` with object metadata
### Low Priority
- [ ] Add comprehensive tests for list operations
- [ ] Add integration tests with different storage backends
- [ ] Document any limitations compared to native object_store implementations
## Implementation Notes
1. **Copy/Rename Operations**: Check if OpenDAL's backend supports native copy/rename. If not, consider fallback to client-side implementation with appropriate warnings.
2. **ETag/Version Support**: Different backends have varying support for ETags and versioning. Implementation should gracefully handle backends that don't support these features.
3. **Attributes Mapping**: Need to define how OpenDAL's metadata maps to object_store's attributes.
## References
- [object_store documentation](https://docs.rs/object_store/latest/object_store/)
- [OpenDAL object_store integration](integrations/object_store/)
## Acceptance Criteria
- All object_store trait methods return meaningful results instead of `NotSupported`
- Metadata (ETag, version, attributes) is properly populated when available
- Comprehensive test coverage for all implemented methods
- Documentation clearly states any limitations or backend-specific behavior
Contributor guide
Research direction
Start in integrations/object_store/src/store.rs, focusing on put_multipart_opts, copy, rename, copy_if_not_exists, and the PutResult and GetResult conversions. Review the object_store trait requirements alongside OpenDAL's supported operations and existing integration behavior. Done means unsupported methods are addressed where possible, metadata is mapped when available, and the listed list and backend integration tests cover the behavior and limitations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100