snowflakedb / snowflakedb/snowflake-connector-python

SNOW-3790287: Pyarrow version warning fires at import time even when pandas tools are never used

Open
#2,950 2 comments 3 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

status-triage_done
Dominant language
Python
Stars
730
Forks
574
Avg merge
5h 45m
Merged PRs (30d)
16

Description

I'm getting below warning on every cold start of my AWS Lambda function, even though I am using just import snowflake.connector and from snowflake.connector import DictCursor in my code. I never call any pandas-related API:

UserWarning: You have an incompatible version of 'pyarrow' installed (24.0.0), please install a version that adheres to: 'pyarrow<19.0.0; extra == "pandas"'

The pyarrow in my environment is pulled in by **awswrangler** — it has nothing to do with the snowflake connector. I am not calling write_pandas, fetch_pandas_all, or any arrow-based API anywhere in my code.

I traced where the warning comes from. options.py line 130 does this at the top level of the module, outside any function:

pandas, pyarrow, installed_pandas = _import_or_missing_pandas_option()

So every time import snowflake.connector runs, it immediately calls _import_or_missing_pandas_option(), which checks whether pyarrow is installed and warns if the version is incompatible. The full chain that triggers it is __init__.py → cursor.py → result_batch.py → options.py:130. There is no way to avoid it short of suppressing warnings manually.

To reproduce, install the packages and run a minimal import — no pandas, no arrow, just a bare connector import:

  1. Install: pip install snowflake-connector-python==3.16.0 pyarrow==24.0.0
    **2. Run: python -W all -c "import snowflake.connector; from snowflake.connector import DictCursor; print('done')"
  2. The warning fires before print('done') even runs.**

Since v3.5.0, pyarrow is no longer a required dependency as the connector switched to nanoarrow internally. That means pyarrow being present in the environment is now completely normal for many users who have nothing to do with snowflake's pandas integration. The check should not run at import time,it should only run inside the pandas-specific functions when they are actually called.

Environment: snowflake-connector-python 3.16.0, pyarrow 24.0.0 (from awswrangler), Python 3.13.7, AWS Lambda.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.