snowflakedb / snowflakedb/snowflake-connector-python

SNOW-649836: insert null into a variant column

Open
#1,232 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature status-triage_done triaged v5.0.0
Dominant language
Python
Stars
730
Forks
574
Avg merge
5h 45m
Merged PRs (30d)
16

Description

What is the current behavior?

I have a variant column with multiple fields inside it, whenever there is a None, it doesnt insert that particular key in variant column and just skips it -

df2 = pd.DataFrame([('Mark', {'K': 2,'Z':None}), ('Luke', {'K': 4,'Z':'bla'})], columns=['ID', 'V'])

when i do write_pandas on it using this

# Write the data from the DataFrame to the table named "customers".
success, nchunks, nrows, _ = write_pandas(ctx, df2, 'customers')

This is how it inserts in snowflake now.

row   ID           V
1        Mark    { "K": 2 }
2        Luke     { "K": 4, "Z": "bla" }

What is the desired behavior?

It should look in snowflake table like this.

row   ID           V
1        Mark    { "K": 2 ,"Z":NULL}
2        Luke     { "K": 4, "Z": "bla" }

How would this improve snowflake-connector-python?

There is no option available to insert null value for a key in variant columns, snowflake allows it thought it interface, but its not possible to do it using write_pandas

References, Other Background

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the write_pandas entry point and reproduce the DataFrame example with a variant column containing a None value. Trace how nested dictionaries are serialized before insertion, then verify that the resulting row preserves the Z key with a NULL value while the existing non-null row remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.