Azure / Azure/azure-sdk-for-rust
Cosmos native ABI truncates embedded NUL in string partition-key components
- Dominant language
- Rust
- Stars
- 884
- Forks
- 365
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 112
Description
## Problem
The Cosmos native ABI represents a string partition-key component as a NUL-terminated C string:
```c
const char *string_value;
```
The native wrapper converts it using `CStr::from_ptr`. Consequently, a logical partition-key value containing an embedded NUL cannot be represented faithfully.
For example, a host binding passing the UTF-8 bytes for `"tenant\0admin"` through a conventional C-string allocation causes the driver to observe only `"tenant"`. The two distinct keys `"tenant"` and `"tenant\0admin"` therefore collapse to the same native input. A request can be routed to the wrong logical partition or return a misleading not-found response.
Relevant surfaces:
- `azure_data_cosmos_driver_native/include/azurecosmosdriver.h`: `cosmos_partition_key_component_t`
- `azure_data_cosmos_driver_native/src/partition_key.rs`: conversion through `CStr::from_ptr`
- inline `partition_key_components` on `cosmos_operation_request_t`
## Cross-SDK behavior
The direct Rust SDK preserves the full string and hashes all UTF-8 bytes. Java, .NET, and Python also use length-aware strings and preserve embedded NUL, typically serializing it as JSON `\u0000`. I found no documented Cosmos DB service restriction forbidding NUL in partition-key strings.
This means the narrowing is specific to the native ABI, not a service contract. Go v2 is currently the only active external SDK binding found using this ABI, but every future Java/.NET/Python native wrapper would inherit the same limitation unless it independently rejects NUL.
## Proposed fix
Represent string components with pointer plus byte length, for example:
```c
struct cosmos_string_view_t {
const uint8_t *data;
uintptr_t len;
};
```
or add a `string_value_len` field beside `string_value` while preserving ABI compatibility as appropriate.
The native wrapper should then construct the Rust string from the complete byte slice, validate UTF-8 without relying on a terminator, and preserve all bytes through partition-key hashing and serialization.
## Tests
Please add native ABI coverage proving:
1. `"tenant"` and `"tenant\0admin"` remain distinct.
2. An embedded-NUL component round-trips into the exact Rust partition-key value.
3. Hierarchical partition keys preserve embedded NUL independently in every string component.
4. Invalid UTF-8 still returns the existing typed ABI error.
## Current Go binding behavior
The Go v2 binding currently uses the ABI as exposed, so this gap is being left for the ABI-level fix rather than defining NUL as an invalid partition-key value only in Go. Once the ABI becomes length-aware, Go can pass the complete string without a compatibility workaround.
Contributor guide
Research direction
Start with azure_data_cosmos_driver_native/include/azurecosmosdriver.h and azure_data_cosmos_driver_native/src/partition_key.rs, then trace the inline partition_key_components field on cosmos_operation_request_t. Review the existing native ABI error handling and tests before choosing the length-aware representation. Done means embedded-NUL and hierarchical components remain distinct and round-trip exactly, while invalid UTF-8 still returns the existing typed ABI error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, rust
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100