rstudio / rstudio/vetiver-python

tracking requirements in `required_pkgs`

Open
#140 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement pins
Dominant language
Python
Stars
71
Forks
20
PR merge metrics
No merged PRs in 30d

Description

This conversation is starting to get lost in #126, so bringing it over here :)

From @juliasilge

Both start out with a base level of just the packages directly required to make a prediction. This is some level of likely to work and possibly be enough, especially in R where updating to latest is basically always the right move. Then both R and Python will have an option to escalate to more robust package version tracking.

In R, we're going straight to renv, since that is the tool most people are familiar with for this type of task, a tool we have input into how it develops, etc. So there are two levels, both familiar to R users: only package names, plus opt in to full renv. In Python, the thing that is most equivalent to renv (pipfile.lock) can seem like overkill and may be less familiar to many practitioners. Instead we can use piptools to generate a requirements.txt that is pinned to specific versions and covers the whole dependency graph. So there are two levels here too, but they are different to be more comfortable for Python users: only package names, plus opt in to the piptools pinned requirements.

The general idea would be that instead of required_pkgs, there would be an argument called requirements or requirements_txt. The default would be what required_pkgs does currently: give the names of the minimal required packages to make predictions at a model's endpoint. There could be another argument that would make this minimal requirements be more robust. The top level requirements would include the version (ie, vetiver==0.1.8 and scikit-learn==1.2.0), and pip-tools would be used to find the second-level compatible version. (There is the issue with just doing pip freeze is that it will include everything in the environment, and maybe more annoyingly, is not a guarantee that the environment can be recreated.)

So,

my_vetiver_model.requirements

could output something like:

vetiver
scikit-learn

or something like below, where it is generated from a pinned vetiver==0.1.8 and scikit-learn==1.2.0:

...
requests==2.28.1
    # via
    #   pins
    #   vetiver
rfc3986[idna2008]==1.5.0
    # via httpx
rsconnect-python==1.13.0
    # via vetiver
scikit-learn==1.2.0
    # via
    #   -r /var/folders/5w/dhznpltj14n3nxr4fybjj8_w0000gn/T/tmp8p4nsqtj.in
    #   vetiver
scipy==1.9.3
    # via scikit-learn
...

CC: @machow @juliasilge

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing required_pkgs behavior and the proposed my_vetiver_model.requirements interface. Review how Python package names are currently collected, then determine how optional pinned requirements generated with pip-tools and requirements.txt should be represented; done means both minimal package names and the more robust pinned form are supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.