dbt-labs / dbt-labs/dbt

[Bug] python model ref indexing on same line breaks parsing

Open
#12,939 0 comments 0 reactions 0 assignees View on GitHub
engine:v1 status:triage type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

```
def model(dbt, session):
spark_df = dbt.ref("some_model_dict_A")
pd_df_a = dbt.ref("some_model_dict_B").toPandas()[["colb","colc"]].reset_index()
result_df = spark_session.createDataFrame(pd_df_a).join(spark_df, on="id")
return result_df
```

some_model_dict_B is **not** detected as ref

```
def model(dbt, session):
spark_df = dbt.ref("some_model_dict_A")
pd_df_a = dbt.ref("some_model_dict_B").toPandas()[["colb","colc"]].reset_index()
result_df = spark_session.createDataFrame(pd_df_a).join(spark_df, on="id")
return result_df
```

some_model_dict_B is detected as ref

### Expected Behavior

some_model_dict_B should correctly be identified as a ref, even with indexing of the df on the same line

### Steps To Reproduce

[test_parser.py](https://github.com/user-attachments/files/27638462/test_parser.py)

- pull dbt and setup according to docs
- replace dbt-core/tests/unit/parser/test_parser.py with the attached test_parser.py
- run tests `hatch run unit-tests`

### Relevant log output

```shell
def test_python_model_subscript_chained_ref(self):
"""Refs chained through subscript access (e.g. ref(...)["col"].method()) must be detected."""
block = self.file_block_for(python_model_subscript_chained_ref, "nested/py_model.py")
self.parser.manifest.files[block.file.file_id] = block.file
self.parser.parse_file(block)
node = list(self.parser.manifest.nodes.values())[0]
print(node)
> self.assertEqual(
node.refs,
[
RefArgs(name="some_model_dict_A"),
RefArgs(name="some_model_dict_B"),
],
)
E AssertionError: Lists differ: [RefA[52 chars]None)] != [RefA[52 chars]None), RefArgs(name='some_model_dict_B', packa[18 chars]one)]
E
E Second list contains 1 additional elements.
E First extra element 1:
E RefArgs(name='some_model_dict_B', package=None, version=None)
E
E - [RefArgs(name='some_model_dict_A', package=None, version=None)]
E ? ^
E
E + [RefArgs(name='some_model_dict_A', package=None, version=None),
E ? ^
E
E + RefArgs(name='some_model_dict_B', package=None, version=None)]

../tests/unit/parser/test_parser.py:1591: AssertionError
```

### Environment

```markdown
- OS: macos 15.6.1
- Python: 3.12.4
- dbt main branch, commit: 99177409b0e2e8fcc6784ab9353df2491cb94187
```

### Which database adapter are you using with dbt?

spark

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.