Azure / Azure/azure-sdk-for-java
[Cosmos Spark] Add missing container property support in Spark Catalog
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 2.2k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 178
Description
## Summary
The Spark Catalog currently supports a subset of Cosmos DB container properties when creating containers via `TBLPROPERTIES`. This issue tracks adding support for the remaining properties.
Related PR: #48349 (added Vector Embedding Policy support)
## Currently Supported
| Property | TBLPROPERTIES Key |
|---|---|
| Partition Key Definition | `partitionKeyPath` |
| Indexing Policy | `indexingPolicy` |
| Default TTL | `defaultTtlInSeconds` |
| Analytical Store TTL | `analyticalStoreTtlInSeconds` |
| Vector Embedding Policy | `vectorEmbeddingPolicy` (added in #48349) |
## Missing Properties
- [ ] **Unique Key Policy** (`UniqueKeyPolicy`) — enforce unique constraints on document fields
- [ ] **Conflict Resolution Policy** (`ConflictResolutionPolicy`) — configure conflict resolution for multi-region writes
- [ ] **Computed Properties** (`ComputedProperty`) — define server-side computed columns
- [ ] **Client Encryption Policy** (`ClientEncryptionPolicy`) — configure field-level encryption
- [ ] **Full Text Policy** (`CosmosFullTextPolicy`) — configure full text search
- [ ] **Change Feed Policy** (`ChangeFeedPolicy`) — configure change feed behavior
## Implementation Notes
Each property follows the same pattern established by the Vector Embedding Policy PR (#48349):
1. Add a constant in `CosmosConstants.scala` (`TableProperties`)
2. Add a getter in `CosmosContainerProperties.scala`
3. Add JSON bridge methods in `SparkModelBridgeInternal.scala`
4. Wire into container creation in `CosmosCatalogCosmosSDKClient.scala` and `CosmosCatalogManagementSDKClient.scala`
5. Add integration tests in `CosmosCatalogITestBase.scala`
Contributor guide
Assessment
This issue has not been assessed yet.