Azure / Azure/azure-sdk-for-rust

Cosmos native ABI truncates embedded NUL in string partition-key components

Closed
#5,223 1 comment 1 reaction 0 assignees View on GitHub
Cosmos
Dominant language
Rust
Stars
884
Forks
365
Avg merge
2d 19h
Merged PRs (30d)
112

Description

## Problem

The Cosmos native ABI represents a string partition-key component as a NUL-terminated C string:

```c
const char *string_value;
```

The native wrapper converts it using `CStr::from_ptr`. Consequently, a logical partition-key value containing an embedded NUL cannot be represented faithfully.

For example, a host binding passing the UTF-8 bytes for `"tenant\0admin"` through a conventional C-string allocation causes the driver to observe only `"tenant"`. The two distinct keys `"tenant"` and `"tenant\0admin"` therefore collapse to the same native input. A request can be routed to the wrong logical partition or return a misleading not-found response.

Relevant surfaces:

- `azure_data_cosmos_driver_native/include/azurecosmosdriver.h`: `cosmos_partition_key_component_t`
- `azure_data_cosmos_driver_native/src/partition_key.rs`: conversion through `CStr::from_ptr`
- inline `partition_key_components` on `cosmos_operation_request_t`

## Cross-SDK behavior

The direct Rust SDK preserves the full string and hashes all UTF-8 bytes. Java, .NET, and Python also use length-aware strings and preserve embedded NUL, typically serializing it as JSON `\u0000`. I found no documented Cosmos DB service restriction forbidding NUL in partition-key strings.

This means the narrowing is specific to the native ABI, not a service contract. Go v2 is currently the only active external SDK binding found using this ABI, but every future Java/.NET/Python native wrapper would inherit the same limitation unless it independently rejects NUL.

## Proposed fix

Represent string components with pointer plus byte length, for example:

```c
struct cosmos_string_view_t {
const uint8_t *data;
uintptr_t len;
};
```

or add a `string_value_len` field beside `string_value` while preserving ABI compatibility as appropriate.

The native wrapper should then construct the Rust string from the complete byte slice, validate UTF-8 without relying on a terminator, and preserve all bytes through partition-key hashing and serialization.

## Tests

Please add native ABI coverage proving:

1. `"tenant"` and `"tenant\0admin"` remain distinct.
2. An embedded-NUL component round-trips into the exact Rust partition-key value.
3. Hierarchical partition keys preserve embedded NUL independently in every string component.
4. Invalid UTF-8 still returns the existing typed ABI error.

## Current Go binding behavior

The Go v2 binding currently uses the ABI as exposed, so this gap is being left for the ABI-level fix rather than defining NUL as an invalid partition-key value only in Go. Once the ABI becomes length-aware, Go can pass the complete string without a compatibility workaround.

Contributor guide

Open the contributing guide

Research direction

Start with azure_data_cosmos_driver_native/include/azurecosmosdriver.h and azure_data_cosmos_driver_native/src/partition_key.rs, then trace the inline partition_key_components field on cosmos_operation_request_t. Review the existing native ABI error handling and tests before choosing the length-aware representation. Done means embedded-NUL and hierarchical components remain distinct and round-trip exactly, while invalid UTF-8 still returns the existing typed ABI error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, rust
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.