microsoft / microsoft/typespec

[C#] Support BinaryData-backed data URI serialization

Open
#11,649 1 comment 2 reactions 0 assignees View on GitHub
emitter:client:csharp feature
Dominant language
Java
Stars
5.9k
Forks
394
Avg merge
1d 23h
Merged PRs (30d)
104

Description

# Problem

The C# client emitter supports Base64 and Base64Url encodings for `bytes`, but it does not support data URIs of the form:

```text
data:;base64,
```

This is distinct from the Base64 JSON value discussed in #11623. A data URI is serialized as a JSON string that contains a media type, the Base64 marker, and the encoded payload. Some service properties also accept either a normal URL or a data URI in the same wire value. Without generator support, libraries generally model these properties as `string` or `url`. A convenience API that accepts `BinaryData` must then create and retain the complete data URI before the client
operation starts.

For large images and files, this creates two large UTF-16 strings:

```csharp
string base64 = Convert.ToBase64String(data.ToMemory().Span);
string dataUri = $"data:{mediaType};base64,{base64}";
```

An attempted C# customization in `openai-dotnet` retained `BinaryData` and created the data URI only when the generated model was serialized. The customization had to replace generated properties and `JsonModelWriteCore` methods across six models:

- Chat image content
- Chat file content
- Responses input image content
- Responses input file content
- Image generation input masks
- Computer screenshot output

This behavior belongs in generated serialization rather than hand-maintained model serializers.

## Suggested change

Add data URI support for encoded `bytes` in the C# emitter. A possible TypeSpec representation is:

```typespec
@encode("data-uri")
scalar DataUriBytes extends bytes;
```

For properties that accept either a URL or embedded data, the encoded bytes could participate in a union:

```typespec
union ImageLocation {
url,
DataUriBytes,
}
```

The exact TypeSpec representation is open for discussion. The important C# behavior is:

- Retain `BinaryData` and its media type without creating the data URI during model construction.
- Serialize a standards-compliant data URI string when writing JSON.
- Do not retain the temporary encoded string after serialization.
- Materialize and cache a string only when a string-valued model property is explicitly read.
- Preserve ordinary URL values without Base64 processing.
- Parse deserialized data URIs when binary access is required.
- Generate documentation that no copy is made and mutable backing memory must remain unaltered until the client operation completes.

Possible C# implementation shape

```csharp
internal sealed class DataUriValue
{
private string? _value;

public DataUriValue(BinaryData bytes, string mediaType)
{
Bytes = bytes;
MediaType = mediaType;
}

public DataUriValue(string value)
{
_value = value;
}

public BinaryData? Bytes { get; }

public string? MediaType { get; }

public string GetValue(bool cache)
{
if (_value is not null)
{
return _value;
}

string value = CreateDataUri(Bytes!, MediaType!);
if (cache)
{
_value = value;
}

return value;
}
}
```

Generated model serialization could use the non-caching path:

```csharp
writer.WritePropertyName("image_url"u8);
writer.WriteStringValue(_imageValue.GetValue(cache: false));
```

A public or customized string property could use the caching path:

```csharp
public string ImageUri
=> _imageValue?.GetValue(cache: true);
```

On .NET 8 and later, data URI creation can write directly into one
final string:

```csharp
private static string CreateDataUri(
BinaryData data,
string mediaType)
{
ReadOnlyMemory memory = data.ToMemory();
const string dataPrefix = "data:";
const string base64Prefix = ";base64,";
int base64Length = checked(((memory.Length + 2) / 3) * 4);
int prefixLength = checked(
dataPrefix.Length
+ mediaType.Length
+ base64Prefix.Length);

return string.Create(
checked(prefixLength + base64Length),
(memory, mediaType, base64Length),
static (destination, state) =>
{
int offset = 0;
dataPrefix.AsSpan().CopyTo(destination);
offset += dataPrefix.Length;
state.mediaType.AsSpan().CopyTo(destination[offset..]);
offset += state.mediaType.Length;
base64Prefix.AsSpan().CopyTo(destination[offset..]);
offset += base64Prefix.Length;

if (!Convert.TryToBase64Chars(
state.memory.Span,
destination[offset..],
out int charsWritten)
|| charsWritten != state.base64Length)
{
throw new InvalidOperationException(
"Base64 encoding did not produce the expected output.");
}
});
}
```

The implementation location and generated model shape are open for discussion. A shared runtime helper may be preferable to emitting the support type into every library.

## Target framework compatibility

The lazy value representation only requires APIs available to `netstandard2.0`.

The single-string encoding path uses
`Convert.TryToBase64Chars`, which is not available for
`netstandard2.0`. The emitter can use conditional compilation:

- .NET 8 and later use `string.Create` and `Convert.TryToBase64Chars`.
- `netstandard2.0` and .NET Framework use the existing array-aware `Convert.ToBase64String(byte[], int, int)` path.
- Non-array-backed memory on older targets retains the existing `ToArray()` fallback.

The lazy value and fallback implementation were compiled with .NET SDK `10.0.302` for both `netstandard2.0` and .NET Framework 4.6.2 with zero warnings. The .NET Framework 4.6.2 build was executed and produced the expected data URI from a sliced `ReadOnlyMemory`.

Compatible fallback

```csharp
private static string CreateDataUri(
BinaryData data,
string mediaType)
{
ReadOnlyMemory memory = data.ToMemory();

#if NET8_0_OR_GREATER
// Single-string implementation shown above.
#else
string base64;
if (MemoryMarshal.TryGetArray(
memory,
out ArraySegment segment)
&& segment.Array is not null)
{
base64 = Convert.ToBase64String(
segment.Array,
segment.Offset,
segment.Count);
}
else
{
base64 = Convert.ToBase64String(memory.ToArray());
}

return $"data:{mediaType};base64,{base64}";
#endif
}
```

## Benchmark

Deferring data URI creation removes payload-dependent allocation from model construction. When serialization is required, creating a single final string reduces end-to-end allocation while avoiding retention of that string by the model. Allocation is reduced by 40 percent for the complete construction and serialization path. Model construction itself becomes independent of payload size.

### Results

| Method | Payload size | Mean | Allocated |
|---|---:|---:|---:|
| Previous eager creation | 1 MB | 1.413 ms | 5.09 MB |
| Previous eager creation and serialization | 1 MB | 2.622 ms | 6.36 MB |
| Lazy model construction | 1 MB | 39.26 ns | 248 B |
| Lazy construction and serialization | 1 MB | 1.206 ms | 3.82 MB |
| Previous eager creation | 10 MB | 12.982 ms | 50.86 MB |
| Previous eager creation and serialization | 10 MB | 23.065 ms | 63.58 MB |
| Lazy model construction | 10 MB | 42.23 ns | 248 B |
| Lazy construction and serialization | 10 MB | 17.708 ms | 38.15 MB |
| Previous eager creation | 100 MB | 160.930 ms | 508.63 MB |
| Previous eager creation and serialization | 100 MB | 237.883 ms | 635.78 MB |
| Lazy model construction | 100 MB | 43.47 ns | 248 B |
| Lazy construction and serialization | 100 MB | 174.924 ms | 381.47 MB |

### Methodology

The benchmarks used BenchmarkDotNet `0.15.8` on .NET `8.0.29` with two warmup iterations and five measurement iterations. The host was Windows 11 on an Intel Xeon Platinum 8370C under Hyper-V. Payloads of 1, 10, and 100 million bytes were created once as `BinaryData`. The eager baseline created the Base64 string and final data URI during model construction. The lazy path retained `BinaryData` and created a non-cached data URI during `ModelReaderWriter` serialization.

Benchmark comparison code

```csharp
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using OpenAI.Chat;
using System.ClientModel.Primitives;
using System.Reflection;

BenchmarkRunner.Run();

[MemoryDiagnoser]
[SimpleJob(warmupCount: 2, iterationCount: 5)]
public class DataUriFactoryBenchmarks
{
private BinaryData _data = null!;
private FieldInfo _imagePartField = null!;
private FieldInfo _imageValueField = null!;
private FieldInfo _cachedValueField = null!;

[Params(1_000_000, 10_000_000, 100_000_000)]
public int PayloadSize { get; set; }

[GlobalSetup]
public void Setup()
{
_data = BinaryData.FromBytes(
new byte[PayloadSize],
"image/png");

_imagePartField = typeof(ChatMessageContentPart)
.GetField(
"_imageUri",
BindingFlags.Instance | BindingFlags.NonPublic)!;

_imageValueField = _imagePartField.FieldType
.GetField(
"_imageValue",
BindingFlags.Instance | BindingFlags.NonPublic)!;

_cachedValueField = _imageValueField.FieldType
.GetField(
"_value",
BindingFlags.Instance | BindingFlags.NonPublic)!;
}

[Benchmark]
public string PreviousEagerCreation()
{
string base64 = Convert.ToBase64String(
_data.ToMemory().Span);

return $"data:image/png;base64,{base64}";
}

[Benchmark(Baseline = true)]
public int PreviousEagerCreationAndSerialization()
{
string base64 = Convert.ToBase64String(
_data.ToMemory().Span);
string dataUri = $"data:image/png;base64,{base64}";

ChatMessageContentPart part =
ChatMessageContentPart.CreateImagePart(
_data,
"image/png");

object imagePart = _imagePartField.GetValue(part)!;
object imageValue =
_imageValueField.GetValue(imagePart)!;
_cachedValueField.SetValue(imageValue, dataUri);

return ModelReaderWriter.Write(part).ToMemory().Length;
}

[Benchmark]
public ChatMessageContentPart LazyFactoryCreation()
{
return ChatMessageContentPart.CreateImagePart(
_data,
"image/png");
}

[Benchmark]
public int LazyFactoryAndSerialization()
{
ChatMessageContentPart part =
ChatMessageContentPart.CreateImagePart(
_data,
"image/png");

return ModelReaderWriter.Write(part).ToMemory().Length;
}
}
```

The virtualization environment may affect absolute timing. Allocation results are the stronger evidence.

## Expected behavior

- TypeSpec can express bytes serialized as a data URI string.
- The C# emitter retains `BinaryData` until serialization rather than eagerly creating a large string.
- Ordinary URL values remain supported for URL-or-data-URI properties.
- Generated serialization does not cache temporary data URI strings.
- Explicit string property access may materialize and cache the value.
- The data URI includes the correct media type and Base64 payload.
- Generated code supports `netstandard2.0`, .NET Framework 4.6.2, and modern target frameworks.
- Generated documentation explains the borrowed-buffer lifetime.
- Existing wire representations remain unchanged.

## Related context

- #11623 covers standard Base64 and Base64Url JSON values. It does not cover data URI strings.
- This was identified while investigating large binary payload memory pressure in [`openai-dotnet#1276`](https://github.com/openai/openai-dotnet/issues/1276).

Contributor guide

Open the contributing guide

Research direction

Start by reading the C# emitter's existing handling of encoded bytes and the related Base64 work in #11623. Determine how the proposed TypeSpec data-uri representation fits URL-or-data unions and generated serialization. Done means lazy BinaryData-backed serialization, correct parsing and documentation, unchanged ordinary URLs, and support for the listed target frameworks.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
compilers, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.