tortoise / tortoise/tortoise-orm

Use Pydantic TypeAdapter for encoding/decoding JSON fields

Open
#1,782 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Is your feature request related to a problem? Please describe.

I am using Postgres JSONB columns for storing complex data structures, and I have Pydantic models representing those data structures. Where the column itself is a Pydantic model, I can use JSONField to encode and decode the model:

  • When encoding, the field uses model_dump to convert the model to a Python object here
  • When decoding, the field uses the __init__ to construct a model from a Python dictionary here

However, some other columns are not themselves Pydantic models but are composite objects that include Pydantic models e.g. list[SomePydanticModel] or dict[str, SomePydanticModel]. I cannot use JSONField for these columns because the instances themselves are not Pydantic models i.e. they do not have the model_dump function and the field type is not ModelMetaclass.

Describe the solution you'd like
Pydantic provides the TypeAdapter class for working with composite objects like this. Its dump_python function behaves like model_dump, and its validate_python function behaves like a model's __init__ function. Using the functions on TypeAdapter would allow JSONField to handle composite models in the same way that it handles simple models.

Describe alternatives you've considered
I currently have an implementation of a Tortoise ORM field that uses TypeAdapter. My implementation bypasses the encoder and decoder functions, opting instead to use dump_json and validate_json directly instead. I intend to open a PR with this implementation, but I'm open to guidance to do something differently to bring this capability into Tortoise ORM.

Additional context
I am not 100% sure of the performance cost of instantiating TypeAdapter, so we may want to include some form of caching constructed instances.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tortoise/fields/data.py at the linked JSONField encoding and decoding logic, then read the Pydantic TypeAdapter documentation for dump_python and validate_python. Compare the proposed encoder/decoder approach with the described dump_json and validate_json alternative, including whether adapter construction needs caching. Done means JSONField supports composite objects containing Pydantic models while preserving existing model handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.