dbt-labs / dbt-labs/dbt-adapters

[Bug] Python Model Error Upgrading to dbt Cloud Versionless: Using snowflake.snowpark.session.create_dataframe() method

Open
#698 4 comments 0 reactions 0 assignees View on GitHub
pkg:dbt-snowflake type:bug type:regression
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-snowflake?

- [X] I believe this is a new bug in dbt-snowflake
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

Recommended by dbt support to raise an issue here:

1. We were running on dbt Cloud 1.7 and testing our upgrade to dbt Cloud versionless
2. No changes were to our dbt project's repo, and we encountered errors on ALL our python models (roughly 10 models)

```
The error states as follows: "raise error_class(
snowflake.snowpark.exceptions.SnowparkSQLException: (1304): 01b7d7e3-0611-d6d2-0028-fb832aac9de2: 090106 (22000): Cannot perform DROP. This session does not have a current schema. Call 'USE SCHEMA', or use a qualified name."
```

I found a workaround, but I wanted to highlight this to you all to see if this is a bug. My workaround was to change the method we use in all our python models, that generates the output dataframe.

- We were using snowflake.snowpark.session.create_dataframe() method
- We switched to pandas.Dataframe() -- which resolved all our errors across all our python models

### Expected Behavior

Expected behavior is for no error to occur, since we were only changing our dbt version from 1.7 to versionless

### Steps To Reproduce

1. Change our environment in dbt Cloud to go from version 1.7 to versionless
2. Run our entire project's DAG (or just the set of python models, to be specific)
3. Errors occur on all our python models with the message

The error states as follows: "raise error_class(
snowflake.snowpark.exceptions.SnowparkSQLException: (1304): 01b7d7e3-0611-d6d2-0028-fb832aac9de2: 090106 (22000): Cannot perform DROP. This session does not have a current schema. Call 'USE SCHEMA', or use a qualified name."

### Relevant log output

```shell
Here's an output of one of the python models that failed. Error message at bottom of the log.

18:17:42
18:17:42 Database Error in model [redacted_model_name] (models/intermediate/core/contracts/transformed/[redacted_model_name].py)
100357 (P0000): Python Interpreter Error:
Traceback (most recent call last):
File "_udf_code.py", line 303, in main
materialize(session, df, dbt.this)
File "_udf_code.py", line 298, in materialize
df.write.mode("overwrite").save_as_table('dbt_dev.STAGING."[redacted_model_name]"', table_type='')
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/telemetry.py", line 195, in wrap
result = func(*args, **kwargs)
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/dataframe_writer.py", line 276, in save_as_table
result = session._conn.execute(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 526, in execute
result_set, result_meta = self.get_result_set(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 207, in wrap
raise ne.with_traceback(tb) from None
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 138, in wrap
return func(*args, **kwargs)
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 657, in get_result_set
self.run_query(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 125, in wrap
raise ex
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 119, in wrap
return func(*args, **kwargs)
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 433, in run_query
raise ex
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 418, in run_query
results_cursor = self.execute_and_notify_query_listener(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/snowpark/_internal/server_connection.py", line 369, in execute_and_notify_query_listener
results_cursor = self._cursor.execute(query, **kwargs)
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/connector/cursor.py", line 1015, in execute
Error.errorhandler_wrapper(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/connector/errors.py", line 232, in errorhandler_wrapper
handed_over = Error.hand_to_other_handler(
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/connector/errors.py", line 287, in hand_to_other_handler
cursor.errorhandler(connection, cursor, error_class, error_value)
File "/usr/lib/python_udf/6d863adfdaffa5e0fdef55e2417de4099289e1d46420e02c249458438fbcbd66/lib/python3.8/site-packages/snowflake/connector/errors.py", line 165, in default_errorhandler
raise error_class(
snowflake.snowpark.exceptions.SnowparkSQLException: (1304): 01b7d7e3-0611-da5c-0028-fb832aacc2de: 090106 (22000): Cannot perform DROP. This session does not have a current schema. Call 'USE SCHEMA', or use a qualified name.
in function [redacted_model_name]_DBT_SP with handler main
compiled code at target/run/windblade/models/intermediate/core/contracts/transformed/[redacted_model_name].py
```

### Environment

```markdown
- OS: N/A
- Python: 3.8 (per the log output)
- dbt-core: N/A
- dbt-snowflake: Unsure, b/c using dbt Cloud
- Snowflake Bundle: 2024_07
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.