dotnet / dotnet/SqlClient

SqlDataReader.WillHaveEnoughData is pessimistic, can it read ahead?

Open
#533 1 comment 0 reactions 0 assignees View on GitHub
Performance :chart_with_upwards_trend:
Dominant language
C#
Stars
989
Forks
340
Avg merge
4d 19h
Merged PRs (30d)
72

Description

This is a nice-to-have idea bourne out of some performance investigation.

When doing calling into `SqlDataReader.ReadAsync` the internal function `WillHaveEnoughData` is called to see if it's possible to read another row from the current packet without needing to do IO, it calculates the maximum size of each field that could be fetched and uses that to work out if there is enough data present. In the case of strings the maximum size may often exceed the real size by a large amount causing preeimistic behaviour and allocation of a TaskCompletionSource where one Is not needed.

If possible it would be nice if `WillHaveEnoughData` could have a way to discover if enough column header information is present in the buffer already to determine the size of the next row and avoid the IO if possible. This may require the ability to peek forwards into the buffer by multiple tokens without consuming or doing io.

In practice this behaviour causes a ReadAsync with any variable length type to assume large amounts of data are needed causing task machinery overheads on all invocations where it is only needed in a fraction of the cases.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.