[API Proposal]: Get/Enumerate line spans from TextReader

Open
#114,987 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
csharp
Domain
api, backend-api-design

Research direction

Start with the System.IO.TextReader API and the span-based Regex APIs referenced in the proposal. Resolve the ReadLineAsSpan, EnumerateLines, and TryReadLine designs, then identify the affected implementation and tests; done means an agreed API design with implementation and coverage for line enumeration and span lifetime behavior.

Written by the indexing model from the issue text.

Description

api-suggestion area-System.IO
Background and motivation

When parsing a file line by line, and there's no need to keep the whole line as a string, span based APIs will be helpful for reducing unnecessary allocations up to the file size, as we already have span based Regex APIs.

API Proposal
namespace System.IO;

public class TextReader
{
    public ReadOnlySpan<char> ReadLineAsSpan();
    public TextReaderLineEnumerator EnumerateLines();
}

public ref struct TextReaderLineEnumerator
{
    // Similar to SpanLineEnumerator
}
API Usage

ReadLineAsSpan:

using var reader = new StreamReader(file);

while (!reader.EndOfStream)
{
    ReadOnlySpan<char> line = reader.ReadLineAsSpan();   // <----- gets the span

    if (line.TrimStart().StartsWith('['))
    {
        // continue parsing with Regex
        foreach (var match in OpenBlockRegex.EnumerateMatches(line))
        {
        }
    }
        // or ignore the line
}

EnumerateLines:

using var reader = new StreamReader(file);

foreach (ReadOnlySpan<char> line in reader.EnumerateLines())
{
    if (line.TrimStart().StartsWith('['))
    {
        // continue parsing with Regex
        foreach (var match in OpenBlockRegex.EnumerateMatches(line))
        {
        }
    }
        // or ignore the line
}
Alternative Designs
/// <returns> false when no more lines can be read (EndOfStream)</returns>
/// <param name="length"> -1 when the span.Length is less than the line</param>
bool TryReadLine(Span<char> destination, out int length);
Risks

For the span returning proposal, subsequent read will invalidate the span, user should not keep the value before the next call.

For the TryReadLine design, user need to manage the buffer by themselves.

Dominant language
C#
Stars
18.3k
Forks
5.6k
PR metrics pending

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/runtime

All issues in dotnet/runtime

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.