apache / apache/opendal

Tracking Copier API

Open
#7,521 0 comments 1 reaction 0 assignees View on GitHub
enhancement releases-note/feat
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

# Summary

Track the remaining work for the `Copier` API proposed by RFC #7519.

# Completed

- [x] RFC: #7519
- [x] Phase 1: API skeleton and one-shot compatibility: #7527
- [x] Phase 2: S3 multipart copier: #7541
- [x] Phase 3: GCS rewrite copier: #7545
- [x] Phase 4: Azure Blob block copier: #7551

# Remaining tasks

## Phase 5: S3-compatible standalone services

- [ ] Evaluate whether `cos`, `oss`, and `obs` can share or adapt the existing S3 multipart copier design.
- [ ] Implement multipart copy for services that expose compatible upload-part-copy APIs.
- [ ] Add service-specific copy multi capabilities only after the protocol path is implemented.

## Phase 6: Monitor-based copy services

- [ ] Evaluate `onedrive` copy, which already starts an async server-side copy and polls a monitor URL internally.
- [ ] If useful, implement a stateful copier that stores the monitor URL and advances on `next()` instead of blocking inside `copy`.
- [ ] Do not expose provider monitor tokens in the public API.

## Phase 7: Remaining one-shot copy services

- [ ] Keep plain one-shot services as-is unless the backend exposes a real long-running or segmented server-side copy primitive.
- [ ] Do not add `copy_can_multi` for client-mediated local copies or provider APIs that only support one-shot copy.

# Constraints

- Keep backend continuation state internal.
- Do not emulate atomic destination conditions such as `if_not_exists` with `stat` plus unconditional copy.
- Prefer backend-native server-side copy primitives over client-mediated read plus write.
- Use `copy_can_multi` only when users can actively request segmented server-side copy through `chunk`.

# References

- RFC PR: #7519

Contributor guide

Open the contributing guide

Research direction

Start with RFC #7519 and the existing S3 multipart copier design, then review the completed phases listed in this issue. Compare the remaining services against their server-side copy capabilities and the constraints around copy_can_multi, chunk, and next(). Done means the applicable copier paths and capability behavior are implemented without exposing monitor state or emulating destination conditions.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, azure, gcp, rust
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.