dotnet / dotnet/runtime

[API Proposal]: Base64Url option to emit padding

Open
#129,847 4 comments 1 reaction 0 assignees View on GitHub
api-ready-for-review api-suggestion area-System.Buffers
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Background and motivation

Base64Url (added in .NET 9, #66841) always emits output without `'='` padding which follows the RFC 7515 (JOSE/JWT) convention. Padding question was raised in #66841 but left unresolved, and the type shipped unpadded.

[RFC 4648](https://datatracker.ietf.org/doc/html/rfc4648) treats the alphabet ([section 5](https://datatracker.ietf.org/doc/html/rfc4648#autoid-10)) and padding (section 3.2) as separate choices, and [section 3.2](https://datatracker.ietf.org/doc/html/rfc4648#autoid-5) leaves padding to the referencing spec. Padded base64url is conformant, and it is the default url-safe encoding elsewhere: Go `base64.URLEncoding`, Python `base64.urlsafe_b64encode`, and Java `Base64.getUrlEncoder()` all pad by default (the unpadded form is the opt-in in each: `RawURLEncoding`, and `withoutPadding()`). For `SHA-256("hello")` those produce `"LPJNul-wow4m6DsqxbninhsWHlwfp0JecwQzYpOLmCQ="`, while `Base64Url.EncodeToString` produces the 43-char `"LPJNul-wow4m6DsqxbninhsWHlwfp0JecwQzYpOLmCQ"`.

When a wire format uses padded base64url, there is no out-of-the-box way to produce it. The workarounds are to call `Base64Url` and append `'='` manually, or to use `Convert.ToBase64*` and substitute `'+'/'/'` with `'-'/'_'`. A concrete case: the [NATS](https://nats.io/) object store stores integrity digests as `"SHA-256="` and derives subject names from padded base64url; [the reference client uses](https://github.com/nats-io/nats.go/blob/36ad6510da3b5b0a2b24fdb6ff3717f150b4fad5/jetstream/object.go#L819) Go's `base64.URLEncoding`, so every other client has to emit padding to interoperate. The NATS .NET client implements this by encoding standard base64 and substituting characters (nats-io/nats.net#1199), exactly the reimplementation #66841 set out to avoid.

Decoding is unaffected: `DecodeFromChars/DecodeFromUtf8` already accept input with or without padding, so this is an encode-side request only. **The reverse is the problem. Other implementations are not all lenient**: Go `base64.URLEncoding.DecodeString` and Python `base64.urlsafe_b64decode` both error when expected padding is missing (Go: `"illegal base64 data at input byte 40"`; Python: `"Incorrect padding"`), whereas Java `Base64.getUrlDecoder` and .NET happen to accept either form. So a .NET producer that emits unpadded output is rejected outright by a Go or Python consumer expecting padded base64url. The receiver can't be relied on to tolerate the difference, which is the reason the producer needs a way to emit padding.

### API Proposal

```csharp
namespace System.Buffers.Text;

[Flags]
public enum Base64UrlOptions
{
None = 0,
InsertPadding = 1,
}

public static partial class Base64Url
{
public static string EncodeToString(ReadOnlySpan source, Base64UrlOptions options);
public static byte[] EncodeToUtf8(ReadOnlySpan source, Base64UrlOptions options);
public static char[] EncodeToChars(ReadOnlySpan source, Base64UrlOptions options);

public static int EncodeToUtf8(ReadOnlySpan source, Span destination, Base64UrlOptions options);
public static int EncodeToChars(ReadOnlySpan source, Span destination, Base64UrlOptions options);

public static bool TryEncodeToUtf8(ReadOnlySpan source, Span destination, out int bytesWritten, Base64UrlOptions options);
public static bool TryEncodeToChars(ReadOnlySpan source, Span destination, out int charsWritten, Base64UrlOptions options);
public static bool TryEncodeToUtf8InPlace(Span buffer, int dataLength, out int bytesWritten, Base64UrlOptions options);

public static OperationStatus EncodeToUtf8(ReadOnlySpan source, Span destination, out int bytesConsumed, out int bytesWritten, Base64UrlOptions options, bool isFinalBlock = true);
public static OperationStatus EncodeToChars(ReadOnlySpan source, Span destination, out int bytesConsumed, out int charsWritten, Base64UrlOptions options, bool isFinalBlock = true);

public static int GetEncodedLength(int bytesLength, Base64UrlOptions options);
}
```

Same pattern as `Base64FormattingOptions` on `Convert.ToBase64String`: a `[Flags]` enum with `None` and one bit, so the default is unchanged and padding is opt-in. On the `OperationStatus` overloads, padding is only emitted on the final block; with `isFinalBlock: false` the option has no effect on that call. Decoding is unchanged, since it already accepts both forms.

### API Usage

```csharp
byte[] hash = SHA256.HashData(data);
string digest = "SHA-256=" + Base64Url.EncodeToString(hash, Base64UrlOptions.InsertPadding);
// "SHA-256=LPJNul-wow4m6DsqxbninhsWHlwfp0JecwQzYpOLmCQ="
```

### Alternative Designs

Leave as-is and have callers append padding or post-process `Convert.ToBase64*`. That works but pushes the same step onto every interop consumer, which is the duplication #66841 set out to remove.

A `bool includePadding` parameter instead of the enum. Fewer new types, but on the `OperationStatus` overloads it sits next to the existing `bool isFinalBlock` and a 5-argument call written as `EncodeToUtf8(source, destination, out consumed, out written, true)` still binds to the current overload, so `true` intended as `includePadding` is taken as `isFinalBlock` and the output is unpadded. That compiles and only fails at the consumer. The enum makes the two arguments non-interchangeable at the call site.

### Risks

Additive. Default behavior is unchanged; padding appears only when explicitly requested.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing System.Buffers.Text.Base64Url encode entry points and the Base64FormattingOptions pattern on Convert.ToBase64String. Check how encoded length, destination sizing, and OperationStatus final-block behavior are currently specified and tested. Done means an approved additive API whose default remains unpadded and whose opt-in form emits padding consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.