snowflakedb / snowflakedb/snowpark-python
SNOW-1648322: ML model registration doesn't work when the package name has dot in it
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 341
- Forks
- 155
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 27
Description
We are trying to log a custom Machine Learning (ML) model to the Snowflake model registry using the log_model method https://docs.snowflake.com/en/developer-guide/snowpark-ml/reference/latest/api/registry/snowflake.ml.registry.Registry
The dependencies are autogluon.tabular and snowflake-ml-python
Below is the code we ran to log the model
custom_model = snowml_registry.log_model(
my_autogluon_model,
model_name="autogluon_model_test",
version_name="version_1",
conda_dependencies=["autogluon.tabular>=1.0", "snowflake-ml-python"],
options={"relax_version": False},
signatures={"predict": predict_sign},
comment='AutoGluon model PoC'
)
The method returns the following error
SnowparkSQLException: (1304): 01b6c2ae-0204-750c-0000-640503c32566: 391546 (XX000): SQL compilation error: Cannot create a Python function with the specified packages. Please check your packages specification and try again. 'Packages not found: - autogluon-tabular[version='>=1.0']'. Hint: These packages are available in other python runtimes:["snowflake-ml-python"->[3.8, 3.10, 3.11], "cloudpickle==2.2.1"->[3.8, 3.10, 3.11]].
The strange part in the error message is that the package that was not found is returned as autogluon-tabular (notice the dash in the name), whereas we pass the correct name of the package autogluon.tabular (with a dot in the name). We suspect that somewhere under the hood the dot in the name of the package is being replaced by a dash and conda is failing to find a package with that name and ultimately failing to log the model.
I found these issues that might be related and/or causing this issue
https://github.com/pypa/pip/issues/3678
https://github.com/pypa/pip/issues/3666
Below are the answers to the generic template for bug reports
Please answer these questions before submitting your issue. Thanks!
-
What version of Python are you using?
This happens in all supported versions of Python in Snowpark (3.8 to 3.11)
-
What operating system and processor architecture are you using?
We used a Snowpark Optimized Warehouse which are based on a Linux OS with x64 architecture, AFAIK.
-
What are the component versions in the environment (
pip freeze)?The package details are provided above
-
What did you do?
The steps we took are pretty much the same as in this quickstart, the only difference is that we use Autogluon instead of PyCaret to log a custom model.
https://quickstarts.snowflake.com/guide/deploying_custom_models_to_snowflake_model_registry/index.html#3 -
What did you expect to see?
I expect the model to successfully register.
-
Can you set logging to DEBUG and collect the logs?
The debug level logs are attached.
error.log
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.