Pluggable JSON serialization/deserialization backend
- Dominant language
- No language data
- Stars
- 188
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [x] I agree to follow Django's Code of Conduct
### Feature Description
Make Django's JSON serialization and deserialization pluggable, so a project can choose which JSON library Django uses internally, the same way it can already choose a password hasher (`PASSWORD_HASHERS` + `django[argon2]`) or a storage backend (`STORAGES`). The standard library `json` module stays the default; switching to another library (e.g. `orjson`, or any future one) is opt-in.
### Problem
Django core calls `json.dumps` / `json.loads` directly in roughly 40 places (`http/response.py`, `core/serializers/json.py`, `core/signing.py`, `contrib/admin`, `contrib/messages`, `contrib/staticfiles`, `db/models/fields/json.py`, `forms/fields.py`, `utils/html.py`, `views/i18n.py`, `test/client.py`, and others). A handful are already customizable per-call (`JsonResponse(encoder=, json_dumps_params=)`, `JSONField(encoder=, decoder=)`, `serialize(cls=...)`, `SESSION_SERIALIZER`, `SERIALIZATION_MODULES`), but most are hardcoded to the stdlib `json` module.
The user-facing surfaces (`JsonResponse`, `json_script`, `JSONField`, `Client`, the serializers) are already pluggable per-call, and a third-party package like [`django-orjson`](https://github.com/adamchainz/django-orjson) by @adamchainz, already covers them opt-in. The genuine gap is the **internal** hardcoded call sites in `contrib/admin`, `contrib/messages`, `contrib/staticfiles`, `views/i18n.py`, `db/models/fields/composite.py`, `forms/utils.py`, etc. Those cannot be swapped from a third-party package without monkey-patching, which is fragile and not something the ecosystem should require of a project with millions of lines of code. The same class of gap was identified for the multipart parser in #105, and resolved there by making the parser pluggable on `HttpRequest`.
### Request or proposal
proposal
### Additional Details
This follows the template established by #105 / PR #20498 (pluggable multipart parser) and the `PASSWORD_HASHERS` + `django[argon2]` precedent: keep the current behavior as the default, define a uniform interface, and let users opt in via a setting rather than per-call overrides.
Per the guidance @carltongibson gave on #105 — define a uniform interface first, default to the current implementation, and treat the opt-in as a stepping stone — the first PRs would introduce **no new setting and no behavior change**: only route the hardcoded `json` calls through a single helper in `django.utils.json` (which already exists in `main` with `normalize_json`, added for `django.tasks` — currently a tiny ~15-line module that step 1 would expand). A later, DEP-gated step would add the opt-in setting and an optional packaging extra.
The proposal is library-agnostic. `orjson` is the most common choice in the ecosystem today, but `msgspec`-based backends (used by [`django-bolt`](https://github.com/dj-bolt/django-bolt) and [`django-rapid`](https://github.com/FarhanAliRaza/django-rapid) by @FarhanAliRaza ) and others could target the same interface. The first implementation does not need to update every internal consumer (admin, messages, staticfiles, etc.) in one go: it only needs to establish the pluggability point, so that a project can install a library and have it picked up project-wide without changing application imports and without a dedicated wrapper package.
The practical difference for a large project is decisive. Today, adopting orjson via `django-orjson` means subclassing `JsonResponse` in every view, changing the test base class in every test module, and passing `serializer=` to every `signing.dumps()` call — potentially thousands of file changes in a large project — and the internal surfaces (admin, messages, staticfiles, i18n) remain unreachable without monkey-patching. Even medium-sized community projects like [djangoproject.com](https://github.com/django/djangoproject.com) (Django's own website, public code) or [Django Packages](https://github.com/djangopackages/djangopackages) would need to change every view import and test base class to adopt orjson today. After step 3, the same adoption is `pip install django[orjson]` plus three lines in `settings.py`: zero view changes, zero test changes, zero signing changes, and the previously unreachable surfaces are covered automatically. Rollback is removing three lines. This is the same push-vs-pull distinction as `PASSWORD_HASHERS` (configure once, framework pulls the backend into every call site) versus subclassing every hasher class manually.
Beyond speed, a non-stdlib backend like orjson also serializes `datetime`, `UUID`, `dataclass`, and `enum.Enum` natively — so `JsonResponse({"created_at": timezone.now()})` works without passing `encoder=DjangoJSONEncoder`. That is a capability gain, not only a ~10x speed gain on `dumps`.
Related:
- #105 — pluggable request body parsers (idea that inspired this approach)
- [django/django#20498](https://github.com/django/django/pull/20498) — merged PR making the multipart parser pluggable on `HttpRequest`
- [Trac #36841](https://code.djangoproject.com/ticket/36841) — Trac ticket for #20498
- [django/deps#88](https://github.com/django/deps/pull/88) — DEP 0017 (Content Type Parsing, renumbered from DEP 0015)
- [DEP 0007](https://github.com/django/deps/blob/main/draft/0007-dependency-policy.rst) — Dependency Policy (governs how optional extras like `django[orjson]` would be added; note: orjson does not support PyPy, but since it's an optional extra, PyPy users simply don't install it and keep the stdlib default)
- #157 — Add project metadata for all optional dependencies (complementary proposal)
- [`SESSION_SERIALIZER`](https://docs.djangoproject.com/en/stable/ref/settings/#std-setting-SESSION_SERIALIZER) — existing per-area serializer setting
- [`SERIALIZATION_MODULES`](https://docs.djangoproject.com/en/stable/ref/settings/#std-setting-SERIALIZATION_MODULES) — existing pluggable serializer registry
- [Forum: "Testing django.tasks and understanding json Encoding"](https://forum.djangoproject.com/t/testing-django-tasks-and-understanding-json-encoding/43876) (Jan 2026) — `TASK_SERIALIZER` discussion where jacobtylerwalls noted consensus for a swappable serializer setting
- [`json_module` parameter in django-modern-rest](https://github.com/wemake-services/django-modern-rest/pull/859) (PR #859) — closest published API contract for a swappable JSON module
- [Trac #17942](https://code.djangoproject.com/ticket/17942) — historical rejection of a global `JSON_RESPONSE_DEFAULT_ENCODER` setting; this proposal differs because it's a backend dict (like `STORAGES`), not a single-value encoder setting
- [`DjangoJSONEncoder`](https://docs.djangoproject.com/en/stable/topics/serialization/#djangojsonencoder) — the encoder whose default behavior changes on opt-in (step 3)
### Implementation Suggestions
A progressive sequence, in the style of #105 / #20498:
1. **Refactor only** — route the hardcoded `json.dumps` / `json.loads` calls in `django/` through `django.utils.json.dumps` / `.loads`. No new setting, no behavior change. Default still stdlib. Delivered as **several small PRs, one per coherent cluster** (admin, serializers, HTTP/test, forms, etc.), each with its own release note, so regressions stay bisectable and reviews tractable. Two surfaces are explicitly **carved out** and stay on stdlib unconditionally, because swapping them is load-bearing rather than a perf win: `core/signing.py` (signed-token determinism depends on `separators=(",", ":")`; orjson does not accept `separators` as a parameter and cannot guarantee byte-identical output, which signed tokens require) and the bare `json.loads(param)` calls in `db/models/fields/json.py` lookup compilation (KeyTransform and friends parse values that came from the database; routing them through a swappable backend would silently change ORM lookup behavior). These calls keep using stdlib `json` directly even after step 3.
2. **Introduce a backend abstraction** — a `BaseJsonBackend` interface (`dumps`, `loads`) with a single `StdlibJsonBackend` implementation that preserves today's behavior exactly (including [`DjangoJSONEncoder`](https://docs.djangoproject.com/en/stable/topics/serialization/#djangojsonencoder) as default, `separators=(",", ":")` in signing, the latin-1 encode step). Still no setting, still stdlib-only. `BaseJsonBackend` is documented as a **public but stable** API: third-party backends may depend on it, so signature changes require a deprecation cycle (the same standard `STORAGES` backends already meet).
3. **Opt-in backend** (DEP-gated) — a `JSON_BACKENDS` setting on the [`STORAGES`](https://docs.djangoproject.com/en/stable/ref/settings/#std-setting-STORAGES) shape (Django 4.2), plus optional packaging extras (e.g. `django[orjson]`) so a project can `pip install django[orjson]` and point the setting at an alternative backend. Per-call `encoder=` / `decoder=` / `cls=` keep working unchanged (the active backend transparently falls back to stdlib for that single call when a custom `JSONEncoder`/`JSONDecoder` is passed, since not every library supports that contract); the opt-in only affects the common path. Two behavior changes on opt-in must be documented in the DEP's "Backwards Compatibility" section: (a) `JSONField(encoder=...)` and `JSONField(decoder=...)` are **never accelerated** by the backend switch — they always fall back to stdlib for correctness, so users should not expect speedup there; (b) `DjangoJSONEncoder` is no longer the implicit default for `JsonResponse`, `json_script`, and the test client when a non-stdlib backend is active, so output for `datetime` / `Decimal` / `UUID` changes to the backend's native formatting. The bundled `OrjsonJsonBackend` ships a `default=` function replicating `DjangoJSONEncoder`'s handling for the types orjson does not cover natively (`Decimal` → `str`, `timedelta` → `duration_iso_string`, `Promise` → `str`), limiting the practical output diff to `datetime` formatting — but it is still an intentional, documented output change that snapshot tests will surface. Step 3 will be accompanied by a **reproducible benchmark and a blog post**, per the contributing guide's "performance changes need benchmarks" rule.
Each step is independently mergeable and backwards-compatible. Step 3 requires a DEP in `django/deps`, on the model of DEP 0017.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with django/utils/json.py and one coherent cluster of the hardcoded JSON call sites listed in the proposal, such as admin or serializers. For the refactor step, route that cluster through the helper without changing behavior, while leaving core/signing.py and the ORM lookup json.loads() calls on the standard library; existing tests should continue to pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100