dotnet / dotnet/runtime

[API Proposal]: MemoryExtensions.CompareTo for ReadOnlySpan<byte> (UTF-8 encoded)

Open
#132,285 6 comments 1 reaction 0 assignees View on GitHub
api-suggestion area-System.Memory
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Background and motivation

With a majority of text files nowadays being UTF-8 encoded, it seems it might be beneficial to have a means to compare two `ReadOnlySpan` instances (UTF-8 encoded), using culture-aware semantics. This would be consistent with the push towards UTF-8 as a first-class citizen.

I was not able to find a pre-existing mechanism for doing so. If one does exist, and I simply missed it, my apologies.

The closest I did find was the following:

```csharp
int MemoryExtensions.CompareTo(this ReadOnlySpan span, ReadOnlySpan other, StringComparison comparisonType);
```

However, this would require first transcoding to UTF-16. In addition to the obvious transcoding overhead, this also potentially explodes memory resource utilization for operations (e.g. sorting) where large numbers of comparisons might be required.

The current options seem to be one of the following: 1) incur transcoding overhead 2) abandon multi-cultural ordering (use ordinal) or 3) abandon platform independence (and P/Invoke something like ucol_strcollUTF8?).

### API Proposal

I propose the addition of a new extension method for `ReadOnlySpan` similar to the pre-existing one for `ReadOnlySpan` (perhaps wrapping ucol_strcollUTF8?).

```csharp
namespace System
{
public static partial class MemoryExtensions
{
public static int CompareTo(this ReadOnlySpan span, ReadOnlySpan other, StringComparison comparisonType);
}
}
```

While I think this is consistent with existing method signatures, I am honestly open to any API surface that solves this same problem. I use this only as an example and do not seek to promote any particular implementation.

### API Usage

```csharp
ReadOnlySpan a = "abc"u8;
ReadOnlySpan b = "def"u8;

int comparison = a.CompareTo(b, StringComparison.CurrentCulture);
```

### Alternative Designs

_No response_

### Risks

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing MemoryExtensions.CompareTo overload for ReadOnlySpan and review the related UTF-8 and culture-aware comparison APIs. Check how platform-specific behavior and tests are handled. Done would require an agreed API shape, implementation approach, and tests, but this proposal leaves those design decisions open.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.