snowflakedb / snowflakedb/snowpark-python
SNOW-893210: NotImplementedError when using SnowflakeFile to read from External Staging in UDF
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 341
- Forks
- 155
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 27
Description
Please answer these questions before submitting your issue. Thanks!
-
What version of Python are you using?
3.8
-
What operating system and processor architecture are you using?
"Linux-5.4.129-200.el7.x86_64-x86_64-with-glibc2.17"
-
What are the component versions in the environment (
pip freeze)? -
What did you do?
CREATE OR REPLACE FUNCTION "LOAD_JSON"(file_path string)
RETURNS variant
LANGUAGE PYTHON
RUNTIME_VERSION = '3.8'
PACKAGES = ('snowflake-snowpark-python')
HANDLER = 'process'
AS
$$
import platform
from snowflake.snowpark.files import SnowflakeFile
import json
def process(file_path):
with SnowflakeFile.open(file_path, 'r') as f:
return json.load(f)
$$;
select "LOAD_JSON"(BUILD_SCOPED_FILE_URL(<<external_blob>>, <<file_path>>));
- What did you expect to see?
Expecting to JSON returned in UDF, but received the following error when calling the UDF:
Traceback (most recent call last):
File "_udf_code.py", line 8, in process
File "/usr/lib/python_udf/fca69f3e27f8e37a6a12047c6ce59553c3e8fbbd8157e24384ef483acdf4ac81/lib/python3.8/json/init.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python_udf/fca69f3e27f8e37a6a12047c6ce59553c3e8fbbd8157e24384ef483acdf4ac81/lib/python3.8/site-packages/snowflake/snowpark/files.py", line 132, in read
raise NotImplementedError(_DEFER_IMPLEMENTATION_ERR_MSG)
NotImplementedError: SnowflakeFile currently only works in UDF and Stored Procedures. It doesn't work locally yet.
in function LOAD_JSON with handler process
It was working until this weekend.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the shown LOAD_JSON UDF with SnowflakeFile.open and inspect snowflake.snowpark.files.SnowflakeFile.read, which raises the reported error. Done means reading a JSON file from external staging in a UDF returns the expected value instead of NotImplementedError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100