litestar-org / litestar-org/polyfactory

Infinite recursion (100% CPU, unbounded RAM) defining a factory for a model field referencing a recursive PEP 695 type alias

Open
#884 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
120
PR merge metrics
No merged PRs in 30d

Description

### Summary

Defining a `ModelFactory` subclass for a pydantic model whose field annotation references a **self-referential PEP 695 type alias** (e.g. `type Json = None | bool | int | float | str | list[Json] | dict[str, Json]`) diverges during **class definition**: `FieldMeta.from_type` expands children with no cycle guard, so the alias is unwrapped forever — 100% CPU and unbounded RAM growth (~2.4 GB in ~45 s in our case). The machine-level symptom is an effectively frozen dev box; the pytest-level symptom is a collection-phase hang that no per-test timeout can kill.

### Environment

- polyfactory **3.3.0** (latest on PyPI at time of writing)
- Python **3.14.7** (Windows 11; the recursion itself is platform-independent)
- pydantic 2.13.x — the model itself validates fine; only factory *definition* diverges

### MCVE

```python
from typing import TypeVar

from pydantic import BaseModel, Field
from polyfactory.factories.pydantic_factory import ModelFactory

type Json = None | bool | int | float | str | list[Json] | dict[str, Json]

ModelT = TypeVar("ModelT", bound=BaseModel)

class BaseSchemaFactory(ModelFactory[ModelT]):
__is_base_factory__ = True

class Document(BaseModel):
metadata: dict[str, Json] = Field(default_factory=dict)

class DocumentFactory(BaseSchemaFactory[Document]):
__model__ = Document

print("never reached")
```

Running this hangs in `__init_subclass__` -> `_check_declared_fields_exist_in_model` -> `get_model_fields` -> `PydanticFieldMeta.from_field_info` -> `FieldMeta.from_type`.

A py-spy dump of the hung process shows an infinitely deep `normalize_type` <-> `from_type` chain:

```
normalize_type (polyfactory\utils\normalize_type.py:57)
(polyfactory\utils\normalize_type.py:57)
normalize_type (polyfactory\utils\normalize_type.py:57) # ... x hundreds of frames
from_type (polyfactory\field_meta.py:158) # ... x dozens of frames
from_field_info (polyfactory\factories\pydantic_factory.py:217)
get_model_fields (polyfactory\factories\pydantic_factory.py:457)
_check_declared_fields_exist_in_model (polyfactory\factories\base.py:1047)
__init_subclass__ (polyfactory\factories\base.py:221)
```

`FieldMeta.from_type` builds a child FieldMeta for every `type_args` member (`field_meta.py:158`), and `normalize_type` unwraps a `TypeAliasType` by returning `type_annotation.__value__` — for a self-referential alias the unwrap produces the same union again (whose args include `list[Json]` / `dict[str, Json]`), so the children expansion never terminates, and every level materializes new type objects (hence the memory blow-up, not just CPU).

### Related work

- #728 fixed recursion for types that use **ForwardRef** — the `TypeAliasType` (PEP 695) path still has no cycle guard.
- The same class of bug existed in litestar's OpenAPI generation (litestar/litestar#4843, self-referential `type JSON` alias -> RecursionError) and was fixed with an in-progress alias guard.
- The hint in #853 ("the fix could be as simple as just not normalizing TypeAliasType types when building FieldMetas") points at the same seam.

### Workaround we currently ship

A `get_model_fields` override in our factory base that substitutes annotations referencing the recursive aliases with a terminating, generatable union (documented as delete-when-fixed). Happy to drop it the moment a cycle guard lands — or to test a fix branch against our real workload: we hit this with a `Json` alias shared across ~60 factory definitions, so it reproduces reliably at scale.

Thanks!

Contributor guide

Open the contributing guide

Research direction

Reproduce the MCVE, then trace field_meta.py:158 through utils/normalize_type.py:57 and the Pydantic field handling in factories/pydantic_factory.py. Compare the existing ForwardRef recursion handling from #728 and verify that a regression test for the self-referential PEP 695 alias completes during factory class definition without unbounded expansion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.