OData / OData/AspNetCoreOData

Json body response encoding of non-ascii characters

Open
#652 3 comments 0 reactions 1 assignee View on GitHub

@xuzhg is already working on this.

Since Aug 2, 2022.

follow-up
Dominant language
C#
Stars
505
Forks
186
PR merge metrics
No merged PRs in 30d

Description

Assemblies affected
At the moment I'm using the latest version of ASP.NET Core OData which has 8.0.10 as version.

Describe the bug
I notice that using the Edm model of an entity and call it from the APIs it converts some characters into the Unicode character instead of representing them.
In my example the oData API response convert the "§" into the Unicode character "\u00a7".

Reproduce steps

  1. Download the following repository https://github.com/federicocodo/oDataTest
  2. Start the WebApplication and call the WeatherForecast API http://localhost:5031/weatherforecast
  3. In the json response you should see that in the "ToDecode" column there is always a "\u00a7" (this corresponds to the "§" character)

Data Model
The Data model of the C# class can be found at the following link https://github.com/federicocodo/oDataTest/blob/master/WebApplicationTestoData/WeatherForecast.cs.

EDM (CSDL) Model
This is the Edm model of the WeatherForecast C# class.

<edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">
    <edmx:DataServices>
        <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="WebApplicationTestoData">
            <EntityType Name="WeatherForecast">
                <Key>
                    <PropertyRef Name="Id"/>
                </Key>
                <Property Name="Id" Type="Edm.Guid" Nullable="false"/>
                <Property Name="Date" Type="Edm.DateTimeOffset" Nullable="false"/>
                <Property Name="TemperatureC" Type="Edm.Int32" Nullable="false"/>
                <Property Name="Summary" Type="Edm.String"/>
                <Property Name="ToDecode" Type="Edm.String" Nullable="false"/>
            </EntityType>
        </Schema>
        <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="Default">
            <EntityContainer Name="Container">
                <EntitySet Name="WeatherForecast" EntityType="WebApplicationTestoData.WeatherForecast"/>
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

Request/Response
The request used for the API call is the same of the previous one write in the steps to reproduce the issue: http://localhost:5031/weatherforecast.
The response of the API call return this json:

{
  "@odata.context": "http://localhost:5031/$metadata#WeatherForecast",
  "value": [
    {
      "Id": "1620ef05-919b-4fb4-8413-f222f3829d5a",
      "Date": "2022-08-09T14:53:52.5736266+02:00",
      "TemperatureC": -13,
      "Summary": "Chilly",
      "ToDecode": "1119601080\u00a71702726918"
    },
    {
      "Id": "4a28017e-1865-4670-9c8b-aea6f482f97a",
      "Date": "2022-08-07T14:53:52.5736213+02:00",
      "TemperatureC": -3,
      "Summary": "Cool",
      "ToDecode": "1226886353\u00a7440318654"
    },
    {
      "Id": "82f3a8f1-887d-4781-828a-56d41564ccfa",
      "Date": "2022-08-06T14:53:52.5728782+02:00",
      "TemperatureC": 53,
      "Summary": "Balmy",
      "ToDecode": "834103851\u00a72146784071"
    },
    {
      "Id": "8af21294-5a0e-41a1-bd06-ac6dc086e8a4",
      "Date": "2022-08-08T14:53:52.5736261+02:00",
      "TemperatureC": 50,
      "Summary": "Bracing",
      "ToDecode": "577055983\u00a71635464599"
    },
    {
      "Id": "fc1b2f7d-2579-4479-b2f9-ddd565c79b15",
      "Date": "2022-08-10T14:53:52.573627+02:00",
      "TemperatureC": 17,
      "Summary": "Bracing",
      "ToDecode": "501313741\u00a7933620329"
    }
  ]
}

Expected behavior
What I expect to see is that in the "ToDecode" column instead of reading "\u00a7" I should read the correct character it represents, from unicode, that is §.

Additional context
During the various tests carried out on oData I noticed how removing the Edm model from the AddRouteComponents in the Program.cs causes the API response json to contain the "§" character not converted to Unicode ("\u00a7") in the "ToDecode" column.
Here the json response:

[
  {
    "id": "31bd8c25-6c6e-453c-8bad-3289f6638988",
    "date": "2022-08-10T16:11:45.3894843+02:00",
    "temperatureC": 52,
    "temperatureF": 125,
    "summary": "Chilly",
    "toDecode": "1760004420§95151189"
  },
  {
    "id": "346158a9-a7ab-43f1-b4e5-e22d58c6cfa2",
    "date": "2022-08-09T16:11:45.389484+02:00",
    "temperatureC": 29,
    "temperatureF": 84,
    "summary": "Cool",
    "toDecode": "164081179§988381653"
  },
  {
    "id": "52315f11-8f0a-4efe-9ee3-faf0fc3e9ff1",
    "date": "2022-08-08T16:11:45.3894835+02:00",
    "temperatureC": -16,
    "temperatureF": 4,
    "summary": "Hot",
    "toDecode": "131708330§1616305195"
  },
  {
    "id": "cd19e3ee-e6e4-4938-9348-5ddeeb3ae7a5",
    "date": "2022-08-06T16:11:45.3889381+02:00",
    "temperatureC": 47,
    "temperatureF": 116,
    "summary": "Mild",
    "toDecode": "1634695148§1250621338"
  },
  {
    "id": "eada80b3-b1a4-4a09-94f5-4e4b7b5ebbe2",
    "date": "2022-08-07T16:11:45.3894805+02:00",
    "temperatureC": 48,
    "temperatureF": 118,
    "summary": "Warm",
    "toDecode": "1998032638§994706651"
  }
]

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.