tortoise / tortoise/tortoise-orm
Conversion to Pydantic model fails when using `source_field`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Describe the bug
When trying to convert a Tortoise model to a Pydantic model with pydantic_model_creator, an KeyError exception is thrown.
To Reproduce
In this example, I try to assign the column name as author_id with source_field, even though I know that adding the suffix _id is the default behaviour of Tortoise. I tried using another name to test, like foo, but the outcome is the same (KeyError is thrown)
from tortoise import fields
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise.models import Model
class User(Model):
id = fields.UUIDField(pk=True)
username = fields.CharField(max_length=16)
follows: fields.ManyToManyRelation["User"] = fields.ManyToManyField(
"models.User",
through="followers",
forward_key="follows_id",
related_name="followers",
)
followers: fields.ReverseRelation["User"]
class Tweet(Model):
id = fields.UUIDField(pk=True)
author: fields.ForeignKeyRelation[User] = fields.ForeignKeyField(
"models.User",
source_field="author_id",
to_field="id",
)
text = fields.CharField(max_length=280)
UserSchema = pydantic_model_creator(User)
TweetSchema = pydantic_model_creator(Tweet)
Versions
- python==3.10.5
- tortoise-orm==0.19.2
- asyncpg==0.26.0
Stacktrace
Traceback (most recent call last):
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/bin/aerich", line 8, in <module>
sys.exit(main())
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/aerich/cli.py", line 257, in main
cli()
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/aerich/cli.py", line 31, in wrapper
loop.run_until_complete(f(*args, **kwargs))
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/aerich/cli.py", line 231, in init_db
await command.init_db(safe)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/aerich/__init__.py", line 134, in init_db
await Tortoise.init(config=self.tortoise_config)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/__init__.py", line 574, in init
cls._init_apps(apps_config)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/__init__.py", line 410, in _init_apps
cls.init_models(info["models"], name, _init_relations=False)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/__init__.py", line 391, in init_models
app_models += cls._discover_models(models_path, app_label)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/__init__.py", line 349, in _discover_models
module = importlib.import_module(models_path)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/johnhancock/Documents/joao/twitter-timeline-broker/app/models/tweet.py", line 19, in <module>
TweetSchema = pydantic_model_creator(Tweet)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/contrib/pydantic/creator.py", line 270, in pydantic_model_creator
field_map_update(included_fields)
File "/Users/johnhancock/Library/Caches/pypoetry/virtualenvs/twitter-timeline-broker-WGLc-ttN-py3.10/lib/python3.10/site-packages/tortoise/contrib/pydantic/creator.py", line 250, in field_map_update
del field_map[raw_field]
KeyError: 'author_id'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tortoise/contrib/pydantic/creator.py, especially field_map_update and the pydantic_model_creator path shown in the traceback. Run the supplied User and Tweet reproduction with ForeignKeyField(source_field="author_id"). Done means conversion completes without the reported KeyError for author_id or an alternate source_field such as foo.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100