snowflakedb / snowflakedb/snowpark-python

SNOW-1648322: ML model registration doesn't work when the package name has dot in it

Open
#2,208 6 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

status-triage_done
Dominant language
Python
Stars
341
Forks
155
Avg merge
4d 16h
Merged PRs (30d)
27

Description

We are trying to log a custom Machine Learning (ML) model to the Snowflake model registry using the log_model method https://docs.snowflake.com/en/developer-guide/snowpark-ml/reference/latest/api/registry/snowflake.ml.registry.Registry

The dependencies are autogluon.tabular and snowflake-ml-python
Below is the code we ran to log the model

custom_model = snowml_registry.log_model(
    my_autogluon_model,
    model_name="autogluon_model_test",
    version_name="version_1",
    conda_dependencies=["autogluon.tabular>=1.0", "snowflake-ml-python"],
    options={"relax_version": False},
    signatures={"predict": predict_sign}, 
    comment='AutoGluon model PoC'
)

The method returns the following error

SnowparkSQLException: (1304): 01b6c2ae-0204-750c-0000-640503c32566: 391546 (XX000): SQL compilation error: Cannot create a Python function with the specified packages. Please check your packages specification and try again. 'Packages not found: - autogluon-tabular[version='>=1.0']'. Hint: These packages are available in other python runtimes:["snowflake-ml-python"->[3.8, 3.10, 3.11], "cloudpickle==2.2.1"->[3.8, 3.10, 3.11]].

The strange part in the error message is that the package that was not found is returned as autogluon-tabular (notice the dash in the name), whereas we pass the correct name of the package autogluon.tabular (with a dot in the name). We suspect that somewhere under the hood the dot in the name of the package is being replaced by a dash and conda is failing to find a package with that name and ultimately failing to log the model.

I found these issues that might be related and/or causing this issue
https://github.com/pypa/pip/issues/3678
https://github.com/pypa/pip/issues/3666

Below are the answers to the generic template for bug reports

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

    This happens in all supported versions of Python in Snowpark (3.8 to 3.11)

  2. What operating system and processor architecture are you using?

    We used a Snowpark Optimized Warehouse which are based on a Linux OS with x64 architecture, AFAIK.

  3. What are the component versions in the environment (pip freeze)?

    The package details are provided above

  4. What did you do?

    The steps we took are pretty much the same as in this quickstart, the only difference is that we use Autogluon instead of PyCaret to log a custom model.
    https://quickstarts.snowflake.com/guide/deploying_custom_models_to_snowflake_model_registry/index.html#3

  5. What did you expect to see?

    I expect the model to successfully register.

  6. Can you set logging to DEBUG and collect the logs?
    The debug level logs are attached.
    error.log

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.