GCP: Support write-time object context annotations in GCS FileIO (equivalent of S3 write.tags)
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
## Background
The AWS module supports attaching arbitrary key-value tags to S3 objects at write time via
`s3.write.tags.=` catalog properties (see `S3FileIOProperties` and `S3OutputStream`).
These tags are embedded in the upload request with no extra API call and can drive S3 lifecycle
policies.
GCS has a direct equivalent called **object contexts** — key-value metadata embedded
during the object write. The GCS Java SDK supports this via
`BlobInfo.Builder.setContexts(ObjectContexts)` since
[commit `485aefd`](https://github.com/googleapis/java-storage/commit/485aefd3047c52c98d8bd913033c8aee1473e988)
(google-cloud-storage ≥ 2.57).
## Proposed change
Add `gcs.write.object-context.=` catalog property support, mirroring the S3 design. This would be useful to add additional metadata like environment, team and table info to the catalog files that are created in GCS.
- **`GCPProperties`** — new constant `GCS_WRITE_OBJECT_CONTEXT_PREFIX = "gcs.write.object-context."`;
constructor collects all matching properties into an immutable `Map writeObjectContexts`;
new accessor `writeObjectContexts()`
- **`GCSOutputStream`** — new private `buildBlobInfo(BlobId, GCPProperties)` that calls
`builder.setContexts(ObjectContexts)` when contexts are configured before passing `BlobInfo` to
`storage.writer()`. No changes to the stream mechanics.
- No changes to `GCSFileIO`, `GCSOutputFile`, or any module outside `gcp/`
## Example usage
```properties
spark.sql.catalog.my_catalog.gcs.write.object-context.env=prod
spark.sql.catalog.my_catalog.gcs.write.object-context.team=data-platform
spark.sql.catalog.my_catalog.gcs.write.object-context.iceberg-table=orders
```
## Related
- S3 write-tags: apache/iceberg#4259
- GCS object contexts SDK commit: [[googleapis/java-storage@485aefd](https://github.com/googleapis/java-storage/commit/485aefd3047c52c98d8bd913033c8aee1473e988)](https://github.com/googleapis/java-storage/commit/485aefd3047c52c98d8bd913033c8aee1473e988)
- GCS object context docs: https://cloud.google.com/storage/docs/use-object-contexts
## AI Disclosure
LLM is used to research the details of S3 write tags and draft the issue description.
### Query engine
None
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start by comparing the existing S3 write-tags implementation with GCPProperties and GCSOutputStream in the gcp module. Check that configured gcs.write.object-context. properties are collected and applied to the BlobInfo before storage.writer() is called; done means GCS writes include the configured contexts without changing stream mechanics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100