services/gcs: XML multipart writer cannot honor write_with_if_not_exists
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
## Summary
GCS supports `write_with_if_not_exists` for single-shot uploads via `ifGenerationMatch=0`, but OpenDAL's multipart writer path uses the XML API multipart upload APIs, which do not support request preconditions.
This was exposed by `test_writer_write_with_if_not_exists` after #8028 added chunked-writer precondition coverage. Against GCS the multipart write with `if_not_exists(true)` succeeds and overwrites an existing object instead of returning `ConditionNotMatch`.
## Evidence from GCS docs
From [Request preconditions](https://docs.cloud.google.com/storage/docs/request-preconditions):
> Preconditions cannot be used in XML API multipart uploads. Attempting to do so results in a `400 NotImplemented` error.
`If-None-Match` is also not a substitute here: GCS documents it for retrieve/GET/HEAD style requests, and the Complete Multipart Upload request headers do not list precondition headers.
## Current OpenDAL behavior
- Single-shot `write_with(...).if_not_exists(true)` works on GCS.
- Capability currently reports `write_with_if_not_exists: true` and `write_can_multi: true`.
- Chunked `writer_with(...).if_not_exists(true)` therefore looks supported, but the multipart path cannot enforce the condition.
## Possible directions
1. Keep skipping `test_writer_write_with_if_not_exists` on GCS until there is a supported upload path.
2. Move GCS multi-write onto an API that can enforce `ifGenerationMatch=0` (for example resumable upload), if that can preserve current writer semantics.
3. Narrow the advertised capability so multipart conditional writes are not implied when only single-shot conditional writes are possible.
## References
- Failing main Behavior Test: https://github.com/apache/opendal/actions/runs/31186355721
- Related conditional-write tracking: #7889
- Azblob multipart precondition fix that introduced the coverage: #8028
Contributor guide
Research direction
Start with test_writer_write_with_if_not_exists and the GCS XML multipart writer and capability implementation; compare the behavior added in #8028 with GCS request-precondition documentation. Determine whether to skip the test, switch to a supported upload path, or narrow the capability, then verify that writes to existing objects return ConditionNotMatch without overwriting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, rust
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100