dart-lang / dart-lang/http

Default charset for "application/json" incorrect

Open
#1,331 3 comments 0 reactions 0 assignees View on GitHub
package:http_parser
Dominant language
Dart
Stars
1.1k
Forks
419
Avg merge
4d 14h
Merged PRs (30d)
8

Description

When a response is received with the Content-Type header set to "application/json" (without specifying the charset) the parser incorrectly assumes the response to be encoded as ISO-8859-1 (Latin-1)
The comments in the code refer to document RFC-2616 and incorrectly concludes the default encoding must be Latin-1. While the RFC indeed mentions a Latin-1 default encoding it does so **only** for **text** responses. No default is assumed for any other media type.

On the other hand, the IANA document describing the "application/json" mediatype (https://www.iana.org/assignments/media-types/application/json) says in its final note:
_No "charset" parameter is defined for this registration.
Adding one really has no effect on compliant recipients._
It is therefore correct **not** to add "charset=utf-8" to the "application/json" Content-Type header.

Finally, RFC-8259 says explicitly that JSON code should always be considered as being Unicode encoded, commonly in UTF-8 format.

Hence from all the above, the correct default charset for "application/json" mediatype must be "utf-8" whether the charset is present or not in the content-type header.
This same reasoning is likely applicable to other mediatype but I didn't research any further.

Contributor guide

Open the contributing guide

Research direction

Locate the HTTP response parser responsible for choosing a charset from Content-Type and inspect the comments referencing RFC-2616. Run the existing parser tests, or find the relevant test area, and add coverage for application/json with and without a charset. Done means JSON is treated as UTF-8 by default without adding a charset parameter to the response header.

Written by the indexing model from the issue text.

Assessment

Tech stack
dart
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.