python / python/cpython

Avoid redundant whitespace scan in json.loads() for documents without surrounding whitespace

Open
#150,860 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance stdlib type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Decoding a JSON document with json.loads() runs a whitespace-skipping regular expression at the start and end of the document through the pure-Python decode() wrapper, on every call, before and after the C scanner does the actual parsing. The overwhelming majority of documents have no leading or trailing whitespace, so both matches scan zero characters yet still pay for the call and the match-object allocation. For the small documents that dominate real traffic, that fixed overhead is a meaningful fraction of the total decode time.

Skipping the leading match when the document does not begin with whitespace, and the trailing match when the parse already consumed the whole string, removes that overhead from the common case. On a tiny document it is roughly 1.5x faster; documents that do have surrounding whitespace keep the original behavior, and large documents are unaffected.

This overlaps with the broader rewrite proposed in gh-117397; it is filed separately so the small, self-contained version can be evaluated on its own.

Linked PRs
  • gh-150861

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at json.loads() and its pure-Python decode() wrapper, then compare the whitespace handling with the C scanner described in the issue and the broader rewrite in gh-117397. Done means documents without surrounding whitespace avoid redundant scans while documents with whitespace retain their behavior; gh-150861 indicates work is already underway.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.