litestar-org / litestar-org/polyfactory
Infinite recursion (100% CPU, unbounded RAM) defining a factory for a model field referencing a recursive PEP 695 type alias
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 120
- PR merge metrics
- No merged PRs in 30d
Description
### Summary
Defining a `ModelFactory` subclass for a pydantic model whose field annotation references a **self-referential PEP 695 type alias** (e.g. `type Json = None | bool | int | float | str | list[Json] | dict[str, Json]`) diverges during **class definition**: `FieldMeta.from_type` expands children with no cycle guard, so the alias is unwrapped forever — 100% CPU and unbounded RAM growth (~2.4 GB in ~45 s in our case). The machine-level symptom is an effectively frozen dev box; the pytest-level symptom is a collection-phase hang that no per-test timeout can kill.
### Environment
- polyfactory **3.3.0** (latest on PyPI at time of writing)
- Python **3.14.7** (Windows 11; the recursion itself is platform-independent)
- pydantic 2.13.x — the model itself validates fine; only factory *definition* diverges
### MCVE
```python
from typing import TypeVar
from pydantic import BaseModel, Field
from polyfactory.factories.pydantic_factory import ModelFactory
type Json = None | bool | int | float | str | list[Json] | dict[str, Json]
ModelT = TypeVar("ModelT", bound=BaseModel)
class BaseSchemaFactory(ModelFactory[ModelT]):
__is_base_factory__ = True
class Document(BaseModel):
metadata: dict[str, Json] = Field(default_factory=dict)
class DocumentFactory(BaseSchemaFactory[Document]):
__model__ = Document
print("never reached")
```
Running this hangs in `__init_subclass__` -> `_check_declared_fields_exist_in_model` -> `get_model_fields` -> `PydanticFieldMeta.from_field_info` -> `FieldMeta.from_type`.
A py-spy dump of the hung process shows an infinitely deep `normalize_type` <-> `from_type` chain:
```
normalize_type (polyfactory\utils\normalize_type.py:57)
(polyfactory\utils\normalize_type.py:57)
normalize_type (polyfactory\utils\normalize_type.py:57) # ... x hundreds of frames
from_type (polyfactory\field_meta.py:158) # ... x dozens of frames
from_field_info (polyfactory\factories\pydantic_factory.py:217)
get_model_fields (polyfactory\factories\pydantic_factory.py:457)
_check_declared_fields_exist_in_model (polyfactory\factories\base.py:1047)
__init_subclass__ (polyfactory\factories\base.py:221)
```
`FieldMeta.from_type` builds a child FieldMeta for every `type_args` member (`field_meta.py:158`), and `normalize_type` unwraps a `TypeAliasType` by returning `type_annotation.__value__` — for a self-referential alias the unwrap produces the same union again (whose args include `list[Json]` / `dict[str, Json]`), so the children expansion never terminates, and every level materializes new type objects (hence the memory blow-up, not just CPU).
### Related work
- #728 fixed recursion for types that use **ForwardRef** — the `TypeAliasType` (PEP 695) path still has no cycle guard.
- The same class of bug existed in litestar's OpenAPI generation (litestar/litestar#4843, self-referential `type JSON` alias -> RecursionError) and was fixed with an in-progress alias guard.
- The hint in #853 ("the fix could be as simple as just not normalizing TypeAliasType types when building FieldMetas") points at the same seam.
### Workaround we currently ship
A `get_model_fields` override in our factory base that substitutes annotations referencing the recursive aliases with a terminating, generatable union (documented as delete-when-fixed). Happy to drop it the moment a cycle guard lands — or to test a fix branch against our real workload: we hit this with a `Json` alias shared across ~60 factory definitions, so it reproduces reliably at scale.
Thanks!
Contributor guide
Research direction
Reproduce the MCVE, then trace field_meta.py:158 through utils/normalize_type.py:57 and the Pydantic field handling in factories/pydantic_factory.py. Compare the existing ForwardRef recursion handling from #728 and verify that a regression test for the self-referential PEP 695 alias completes during factory class definition without unbounded expansion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100