SNOW-788985: Timezone gets removed when creating a DataFrame

Open
#781 1 comment 0 reactions 1 assignee View on GitHub

@sfc-gh-ashahi is already working on this.

Since Mar 20, 2024.

Assessment

This issue has not been assessed yet.

Description

bug triaged

Bug Description

The code snippets shown assume that a Snowpark session is already created

Pandas

from datetime import datetime


def cv(*args):
    return [datetime.strptime(t, "%Y-%m-%d %H:%M:%S%z") for t in args]


df1 = pd.DataFrame({"TIME": cv("2021-01-01 23:00:00+01:00", "2021-01-02 23:00:00+02:00")})
display(df1)
df2 = session.createDataFrame(df1).to_pandas()
display(df2)

The python code above will output the result shown below. The timezones of 'df2' are removed, which shouldn't happen.
df1: '2021-01-01 23:00:00+01:00', '2021-01-02 23:00:00+02:00'
df2: '2021-01-01 22:00:00', '2021-01-02 21:00:00'

When schema is specified

from snowflake.snowpark.types import (
    StructField,
    StructType,
    TimestampType,
)

schema = StructType([StructField("a", TimestampType())])
df = session.create_dataframe(
    [["2021-01-01 23:00:00+01:00"], ["2021-01-02 23:00:00+02:00"]],
    schema,
).to_pandas()
display(df)

When specifying the schema, the only type of Timestamp available is 'TimestampType', which also removes the timezone:
Displays: '2021-01-01 23:00:00', '2021-01-02 23:00:00'

When the schema is inferred, the date gets saved as a string.

Dominant language
Python
Stars
341
Forks
155
Avg merge
4d 16h
Merged PRs (30d)
27

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from snowflakedb/snowpark-python

All issues in snowflakedb/snowpark-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.