dotnet / dotnet/efcore

Enable optimized JSON UTF8 reading/writing

Open
#31,216 1 comment 3 reactions 0 assignees View on GitHub
area-json area-perf area-query area-save-changes
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

Our JSON reading/writing uses Utf8JsonReader/Writer, but currently always does so over UTF16 strings. In other words, for reading we get a string out of DbDataReader, and then encode it into UTF8, build a MemoryStream over that, and then read from that stream (we do something similarly inefficient for writing).

For SQL Server, where a [UTF8 property](https://learn.microsoft.com/en-us/ef/core/providers/sql-server/columns?tabs=ef-core-7#unicode-and-utf-8) is configured, we should be able to get binary data directly out of SqlClient, possibly in the form of a stream. Assuming that's the case, we can parse JSON directly out of that stream without the current UTF16 conversions and copying.

We can also consider making string properties UTF8 by convention on SQL Server when they're a JSON container (but consider the breaking change and how the transition would work). Note also that UTF8 support was introduced in SQL Server 2019.

For PostgreSQL, UTF8 is configured at the database level, and is also the default; we should enable this optimization there as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.