dbt-labs / dbt-labs/dbt-adapters
[Bug] Index errors when using split_part
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this a new bug in dbt-spark?
- [X] I believe this is a new bug in dbt-spark
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
When called with a part index that is out of bounds, and ansi-mode on, the split_part macro leads to an exception
### Expected Behavior
Per the tests in BaseSplitPart in the adapter tests, the expectation is that this macro can be invoked with part indexes greater than the number of parts generated without throwing an exception specifically this row in the seed:
```
,|,,,,
```
We can accommodate this behavior by using get, rather than indexing the array, but only in Spark 3.4.0 or later.
### Steps To Reproduce
1. set spark.sql.ansi.enabled=true
2. use split_part passing an out of bounds index
3. observe exception
### Relevant log output
_No response_
### Environment
This issue has been in there for a while, but I'm just hitting it now due to new defaults in a Databricks environment I was asked to test against.
### Additional Context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.