heroku / heroku/heroku-buildpack-python

Use pip's built-in caching mechanism to improve repeat builds

Open
#926 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Ruby
Stars
993
Forks
1.8k
Avg merge
10h 50m
Merged PRs (30d)
10

Description

12-factor applications should be robust to redeploy, and take steps to boot quickly. This is broken in #909, but after resolving that bug, are there improvements to be made?

Pip by default caches the packages it installs: https://pip.pypa.io/en/stable/reference/pip_install/#caching

But this is turned off for the buildpack currently:
https://github.com/heroku/heroku-buildpack-python/blob/master/bin/steps/pip-install#L50

We do try to cache requirements files and validate updates ourselves on repeat builds:
https://github.com/heroku/heroku-buildpack-python/blob/master/bin/steps/pip-uninstall

But it's built on tooling that has not been maintained:
https://github.com/heroku-python/pip-pop/blob/master/bin/pip-diff

Rather than maintain it by default, investigate what pip supports to see if better tools are now available:
https://pip.pypa.io/en/stable/reference/pip_install/#wheel-cache
https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
https://pip.pypa.io/en/stable/reference/pip_install/#hashes-from-pypi
https://pip.pypa.io/en/stable/reference/pip_check/

- [ ] Proof of concept to investigate if caching-dir will support heroku path requirements (build in temp, copy to `/app/.heroku/*`)
- [ ] Speed test - installing from wheel cache faster than installing from pypi?
- [ ] Security review - more secure to cache wheels?
- [ ] Proof of concept - do repeat builds in ephemeral containers allow for hashing?
- [ ] Proof of concept/Security review - what do we gain checking pypi's hashes against packages?
- [ ] Proof of concept - Verify installed packages have compatible dependencies (which we don't currently do)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.