snowflakedb / snowflakedb/snowflake-connector-python
SNOW-2216201: Allow for passing pyarrow schema to write_pandas
Open
@sfc-gh-snow-drivers-warsaw-dl is already working on this.
Since Jul 23, 2025.
feature
status-triage_done
- Dominant language
- Python
- Stars
- 730
- Forks
- 574
- Avg merge
- 5h 45m
- Merged PRs (30d)
- 16
Description
What is the current behavior?
I have a dataframe where one column holds python dicts. When I want to import it to snowflake using write_pandas it fails on memory (basically eats up all the memory and then fails on malloc).
To solve this I have to define a custom pyarrow schema. However I am unable to pass it as a kwarg to write_pandas because schema is already a named parameter.
Here's the complete flow:
pandas_tools.py
def write_pandas
chunk.to_parquet(chunk_path, compression=compression, **kwargs) # <-- if I pass schema=my_pyarrow_schema here, it works fine, but I cant because write_pandas already has a parameter named schema
What is the desired behavior?
Be able to pass pyarrow schema to write_pandas
How would this improve snowflake-connector-python?
Allow for non-standard dfs to be imported into SF.
References and other background
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.