Adjust API surface for stopping parsing after the first invalid character
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Issue
https://github.com/dotnet/runtime/issues/87171 was approved to be a general purpose way to extend a previously `Utf8Parser` unique feature to support UTF-16 and all number styles rather than the limited subset that `Utf8Parser` covered. In particular this was the ability to parse up until the first invalid character and then succeed rather than fail, allowing for incremental parsing to occur which can be particularly useful for things like CSV, tokenizers, serialization formats (e.g. JSON, YAML, XML, ...), etc.
However, it turns out that doing it this way is a breaking change to F# namely because they allow the following which binds against the existing single out overload and produces a tuple `(bool, int)` containing the result type and out parameters:
```fsharp
match Int32.TryParse(rightMost, NumberStyles.Integer, CultureInfo.InvariantCulture) with
| true, n -> n
| false, _ -> 0
```
This F# feature actually works for any shape of `T Method(out U result)` including multiple out parameters. However, it only works if you only have a single overload with out parameters and errors with an ambiguity if you have multiple, such as the approved API surface is introducing.
Due to the way the F# type inferrence works (largely linear), this is not something that's readily feasible for them to fix. We might be able to use ORPA, but F# would first have to add support for ORPA and that may have unintended future impact, particularly since we're not really deprecating or preferring one API over the other.
### Proposal
Given the above and that there was also the discussion of extending this support to other non-numeric types in the future (like `Guid` or `DateTime` which does not take a `NumberStyles`), it is proposed we change the approved API surface to avoid this ambiguity and land on a unified design.
The main proposal is to split this apart into two distinct interfaces that expose the "incremental parsing" nature. The natural term for this in computer science tends to be a "scanner" in contrast to a "parser" so that is the proposed name below. I expect this name may be slightly contentious and we'll need to discuss other names as well. Some other words that may fit are `Consume`, `ParsePrefix`, `ParseLeading`, `ParsePartial`, `Read`, `Take`, or `Tokenize`; but I feel like most of these aren't as "clear".
We could notably get away with only providing the span versions, we don't strictly need the string one (only differing in behavior for `null` as its slightly different from `empty` in exception for parse).
We could also notably get away with only providing the `Try*` APIs or only providing `Scan` either in the form of `int Scan(..., out T result)` or `T Scan(..., out int elementsConsumed)` where a consumption amount of 0 or -1 indicates "failure". But those feel slightly less consistent with normal .NET conventions.
This new shape would solve the question of how it works for non-numeric types as well; provided we're okay with whatever name we land on.
```csharp
namespace System
{
public interface IScannable
where TSelf : IScannable
{
public static abstract TSelf Scan(string s, IFormatProvider? provider, out int charsConsumed);
public static abstract bool TryScan(string? s, IFormatProvider? provider, out TSelf result, out int charsConsumed);
}
public interface ISpanScannable
where TSelf : ISpanScannable
{
public static abstract TSelf Scan(ReadOnlySpan s, IFormatProvider? provider, out int charsConsumed);
public static abstract bool TryScan(ReadOnlySpan s, IFormatProvider? provider, out TSelf result, out int charsConsumed);
}
public interface IUtf8SpanScannable
where TSelf : IUtf8SpanScannable
{
public static abstract TSelf Scan(ReadOnlySpan utf8Text, IFormatProvider? provider, out int bytesConsumed);
public static abstract bool TryScan(ReadOnlySpan utf8Text, IFormatProvider? provider, out TSelf result, out int bytesConsumed);
}
}
namespace System.Numerics
{
public partial interface INumberBase
: IScannable,
ISpanScannable,
IUtf8SpanScannable
{
public static abstract TSelf Scan(string s, System.Globalization.NumberStyles style, IFormatProvider? provider, out int charsConsumed);
public static abstract TSelf Scan(ReadOnlySpan s, System.Globalization.NumberStyles style, IFormatProvider? provider, out int charsConsumed);
public static virtual TSelf Scan(ReadOnlySpan utf8Text, System.Globalization.NumberStyles style, IFormatProvider? provider, out int bytesConsumed);
public static abstract bool TryScan(string? s, System.Globalization.NumberStyles style, IFormatProvider? provider, out TSelf result, out int charsConsumed);
public static abstract bool TryScan(ReadOnlySpan s, System.Globalization.NumberStyles style, IFormatProvider? provider, out TSelf result, out int charsConsumed);
public static virtual bool TryScan(ReadOnlySpan utf8Text, System.Globalization.NumberStyles style, IFormatProvider? provider, out TSelf result, out int bytesConsumed);
}
}
````
We'd then remove the previously approved API surface, including the `AllowTrailingInvalidCharacters` number style.
Contributor guide
Assessment
This issue has not been assessed yet.