dbt-labs / dbt-labs/dbt-adapters

[ADAP-454] [Bug] `api.Column.string_type` causes error

Open
#507 5 comments 0 reactions 0 assignees View on GitHub
pkg:dbt-spark type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-spark?

- [X] I believe this is a new bug in dbt-spark
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

Because there is no overwritten function, it uses a the core class, which is `character varying` which doesn't exist in spark (or databricks), so this api call just errors in spark/databricks targets.

### Expected Behavior

Model to pass, likely using `string` as the type.

### Steps To Reproduce

The easiest way to see it is with a basic model that just casts something to the string type:

```
{{
config(
materialized = 'table',
)
}}

select
cast(10 as {{api.Column.string_type(4000)}}) as test_col
```

Which in the case of a databricks target leads to the following compiled code
```

select
cast(10 as character varying(4000)) as test_col
```

and a `dbt run` gives the error output
```
dbt run --target databricks
10:05:06 Running with dbt=1.4.5
10:05:06 Unable to do partial parsing because config vars, config profile, or config target have changed
10:05:06 Unable to do partial parsing because a project dependency has been added
10:05:07 Found 1 model, 0 tests, 0 snapshots, 0 analyses, 374 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
10:05:07
10:08:17 Concurrency: 1 threads (target='databricks')
10:08:17
10:08:17 1 of 1 START sql table model dbt_ryan.test_models .............................. [RUN]
10:08:17 1 of 1 ERROR creating sql table model dbt_ryan.test_models ..................... [ERROR in 0.64s]
10:08:18
10:08:18 Finished running 1 table model in 0 hours 3 minutes and 10.96 seconds (190.96s).
10:08:18
10:08:18 Completed with 1 error and 0 warnings:
10:08:18
10:08:18 Runtime Error in model test_models (models/test_models.sql)
10:08:18
10:08:18 [PARSE_SYNTAX_ERROR] Syntax error at or near 'varying': missing ')'.(line 19, pos 25)
10:08:18
10:08:18 == SQL ==
10:08:18 /* {"app": "dbt", "dbt_version": "1.4.5", "dbt_databricks_version": "1.4.2", "databricks_sql_connector_version": "2.3.0", "profile_name": "default", "target_name": "databricks", "node_id": "model.dbt_demo.test_models"} */
10:08:18
10:08:18
10:08:18
10:08:18 create or replace table `dbt_ryan`.`test_models`
10:08:18
10:08:18
10:08:18 using delta
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18 as
10:08:18
10:08:18
10:08:18 select
10:08:18 cast(10 as character varying(4000)) as test_col
10:08:18 -------------------------^^^
10:08:18
10:08:18
10:08:18
```

### Relevant log output

_No response_

### Environment

```markdown
- OS: MacOS Ventura 13.3.1 (22E261)
- Python: 3.9.13
- dbt-core: 1.4.5
- dbt-spark: 1.4.1
```

### Additional Context

Issues in BQ: https://github.com/dbt-labs/dbt-adapters/issues/590

dbt-core relevant issue: https://github.com/dbt-labs/dbt-adapters/issues/81

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.