finos / finos/rune-python-runtime

resolve_references does not traverse list members — @ref inside list elements remain unresolved

Open
#37 0 comments 0 reactions 1 assignee Claimed by @plamen-neykov View on GitHub
bug
Dominant language
Python
Stars
0
Forks
3
PR merge metrics
No merged PRs in 30d

Description

## Bug Report

### `resolve_references` does not traverse list members — `@ref` inside list elements remain unresolved

### Steps to Reproduce

1. Clone the repository and set up the development environment:

```bash
git clone https://github.com/regnosys/rune-python-runtime
cd rune-python-runtime
./dev_clean_setup.sh
source .pydevenv/bin/activate
```

2. Run the following self-contained script (no external dependencies beyond the runtime itself):

```python
import json
from typing import List
from typing_extensions import Annotated
from pydantic import Field
from rune.runtime.base_data_class import BaseDataClass

# A keyed type — instances can be registered under an external key
class Party(BaseDataClass):
_ALLOWED_METADATA = {'@key:external'}
name: str = Field(..., description='party name')

# A type whose partyReference field is a @ref to a Party
class Counterparty(BaseDataClass):
role: str = Field(..., description='role')
partyReference: Annotated[
Party,
Party.serializer(),
Party.validator(allowed_meta=('@ref:external',))
] = Field(..., description='reference to party')
_KEY_REF_CONSTRAINTS = {'partyReference': {'@ref:external'}}

# Trade holds parties (the keys) and counterparties (the refs) both as lists
class Trade(BaseDataClass):
party: List[Party] = Field(..., description='parties')
counterparty: List[Counterparty] = Field(..., description='counterparties')

data = json.dumps({
"party": [
{"@key:external": "p1", "name": "Party A"},
{"@key:external": "p2", "name": "Party B"}
],
"counterparty": [
{"role": "Party1", "partyReference": {"@ref:external": "p1"}},
{"role": "Party2", "partyReference": {"@ref:external": "p2"}}
]
})

# Step A — inspect reference type after deserialize with validate_model=False
t = Trade.rune_deserialize(data, validate_model=False)
print("counterparty[0].partyReference:", type(t.counterparty[0].partyReference).__name__)
print("counterparty[1].partyReference:", type(t.counterparty[1].partyReference).__name__)

# Step B — deserialize with validate_model=True (default)
t2 = Trade.rune_deserialize(data, validate_model=True)
```

### Expected Result

**Step A**: both `partyReference` fields are resolved to `Party` instances after `rune_deserialize`.

**Step B**: deserialization completes without error.

### Actual Result

**Step A** — references are not resolved:

```
counterparty[0].partyReference: UnresolvedReference
counterparty[1].partyReference: UnresolvedReference
```

**Step B** — `ValidationError` is raised because `validate_attribs` re-runs `model_validate` on the already-constructed model and Pydantic rejects the remaining `UnresolvedReference` objects:

```
pydantic.ValidationError: 2 validation errors for Trade
counterparty.0.partyReference
Expected either or dict but got .
[type=Input Validation Error, input_type=UnresolvedReference]
counterparty.1.partyReference
Expected either or dict but got .
[type=Input Validation Error, input_type=UnresolvedReference]
```

### Root Cause

`BaseDataClass.resolve_references` recurses only into properties that are direct `BaseDataClass` instances. It does not descend into list members:

```python
# base_data_class.py — resolve_references
if recurse:
for prop_nm, obj in self.__dict__.items():
if (isinstance(obj, BaseDataClass) # ← True for scalar fields only
and not prop_nm.startswith('__')):
obj.resolve_references(...)
# list-valued properties are silently skipped
```

`Trade.counterparty` is a `list` — `isinstance(list, BaseDataClass)` is `False`, so the `Counterparty` items inside it are never visited and their `partyReference` fields remain as `UnresolvedReference`.

### Environment

- `rune-python-runtime` version: 2.2.0
- Python: 3.11
- OS: macOS Darwin 25.6.0

### Additional Context

**Proposed fix** — extend the recursion in `resolve_references` to traverse list members:

```python
if recurse:
for prop_nm, obj in self.__dict__.items():
if prop_nm.startswith('__'):
continue
if isinstance(obj, BaseDataClass):
obj.resolve_references(ignore_dangling=ignore_dangling,
recurse=recurse)
elif isinstance(obj, list):
for item in obj:
if isinstance(item, BaseDataClass):
item.resolve_references(ignore_dangling=ignore_dangling,
recurse=recurse)
```

**Impact** — any Rune model JSON that uses `@ref` inside a list-valued field is affected:

- `validate_model=False` — reference silently remains `UnresolvedReference`; downstream attribute access fails at runtime
- `validate_model=True` (default) — `ValidationError` raised immediately, preventing use of the deserialized object

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.