snowflakedb / snowflakedb/snowflake-connector-python
SNOW-846438: Getting 0 rows from a TIMESTAMP_TZ column gives timezone-less pandas dtype
Open
Nobody has claimed this yet.
bug
status-in_progress
status-triage_done
triaged
- Dominant language
- Python
- Stars
- 730
- Forks
- 574
- Avg merge
- 5h 45m
- Merged PRs (30d)
- 16
Description
Python version
Python 3.8.16 (default, Mar 1 2023, 21:19:10) [Clang 14.0.6 ]
Operating system and processor architecture
macOS-10.16-x86_64-i386-64bit
Installed packages
aiobotocore==2.5.0
aiohttp==3.8.4
aioitertools==0.11.0
aiosignal==1.3.1
appnope==0.1.3
asn1crypto==1.5.1
asttokens==2.2.1
async-timeout==4.0.2
attrs==23.1.0
backcall==0.2.0
bleach==6.0.0
boto3==1.26.148
botocore==1.29.153
cachetools==5.3.1
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==2.1.1
cloudpickle==2.0.0
contourpy==1.0.7
cryptography==40.0.2
cycler==0.11.0
db-dtypes==1.1.1
decorator==5.1.1
docutils==0.20.1
duckdb==0.8.0
exceptiongroup==1.1.1
execnet==1.9.0
executing==1.2.0
filelock==3.12.0
fonttools==4.39.4
frozenlist==1.3.3
fsspec==2023.6.0
google-api-core==2.11.0
google-auth==2.19.1
google-cloud-bigquery==3.11.0
google-cloud-core==2.3.2
google-crc32c==1.5.0
google-resumable-media==2.5.0
googleapis-common-protos==1.59.0
grpcio==1.54.2
grpcio-status==1.54.2
idna==3.4
importlib-metadata==6.6.0
importlib-resources==5.12.0
iniconfig==2.0.0
ipdb==0.13.13
ipython==8.12.2
jaraco.classes==3.2.3
jedi==0.18.2
jmespath==1.0.1
joblib==1.2.0
keyring==23.13.1
kiwisolver==1.4.4
markdown-it-py==2.2.0
matplotlib==3.7.1
matplotlib-inline==0.1.6
mdurl==0.1.2
-e git+ssh://git@github.com/mvashishtha/modin.git@eeb410c87aa657d556d3cf2403195bdc76cc7193#egg=modin
more-itertools==9.1.0
multidict==6.0.4
Nuitka==1.6.1
numpy==1.24.3
ordered-set==4.1.0
oscrypto==1.3.0
packaging==23.1
pandas==1.5.3
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.5.0
pkginfo==1.9.6
pluggy==1.0.0
prompt-toolkit==3.0.38
proto-plus==1.22.2
protobuf==4.23.2
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==10.0.1
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pycryptodomex==3.18.0
Pygments==2.15.1
PyJWT==2.7.0
pyOpenSSL==23.2.0
pyparsing==3.0.9
pytest==7.3.1
pytest-mock==3.10.0
pytest-xdist==3.3.1
python-dateutil==2.8.2
pytz==2023.3
readme-renderer==37.3
requests==2.31.0
requests-toolbelt==1.0.0
rfc3986==2.0.0
rich==13.4.1
rsa==4.9
s3fs==0.4.2
s3transfer==0.6.1
schedule==1.2.0
scikit-learn==1.2.2
scipy==1.10.1
six==1.16.0
snowflake-connector-python==3.0.3
stack-data==0.6.2
threadpoolctl==3.1.0
tomli==2.0.1
traitlets==5.9.0
twine==4.0.2
typing_extensions==4.6.3
urllib3==1.26.16
wcwidth==0.2.6
webencodings==0.5.1
wrapt==1.15.0
yarl==1.9.2
zipp==3.15.0
zstandard==0.21.0
What did you do?
# con is a snowflake.connector.connection.SnowflakeConnection
cursor = con.cursor()
# first select all the rows
cursor.execute("""SELECT '2021-01-01 00:00:00 +0000'::timestamp_tz AS TIME""")
# i get dtypes with timezone: TIME datetime64[ns, America/Los_Angeles]
print(cursor.fetch_pandas_all().dtypes)
# now filter out the only record.
cursor.execute("""SELECT * FROM (SELECT '2021-01-01 00:00:00 +0000'::timestamp_tz AS TIME) WHERE TIME IS NULL""")
print(cursor.fetch_pandas_all().dtypes)
# Now I lose the timezone. TIME datetime64[ns]
What did you expect to see?
This experience doesn't match the pandas experience, where filtering out all the records from a column should preserve the column's dtype. I want to get datetime64[ns, America/Los_Angeles] (my session time zone) even when my query returns non records.
Can you set logging to DEBUG and collect the logs?
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.