AcademySoftwareFoundation / AcademySoftwareFoundation/rez
better cleanup of failed variant installs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 374
- Avg merge
- 9d 12h
- Merged PRs (30d)
- 5
Description
Currently a variant install/release goes something like this:
PackageRepository.pre_variant_install()is called, which creates a file like<repopath>/mypackage/.building1.0.0. This file indicates that mypackage-1.0.0 may be in the process of being installed. When rez sees this, it doesn't assume that the presence of a1.0.0directory means that that package exists - it checks the package.py and properly verifies (if package.py is present, then the install was successful, because this is done last).- After the variant is installed, this
.buildingfile is removed. - If the variant failed to install (failed build, or failed tests), the
.buildingfile is left in place.
The problem with this is, for newly installed packages, a failed variant install will leave the invalid 1.0.0 directory behind. This doesn't cause a problem, because the .building file remains as well, but it forces rez to do further checks and it pollutes the package repo with incomplete releases.
We need to update this process in a way that deletes these incomplete package version dirs, but in a way that is safe if multiple rez procs were installing variants of the same package at the same time. One way this might work is like so (note that this suggestion is to go along with the existing .building file behaviour, not to replace it):
- When a new package is installed, a lockfile is acquired by the current proc.
- During this lock, the version dir is created. Within the version dir, another file is created, which is unique to this proc (perhaps just a UUID, or based on hostname+pid).
- If it comes time to cleanup a failed variant install, the following occurs:
- we acquire the same lockfile again
- we remove our unique proc file
- if there are no unique proc files left, we delete the package version dir
- On successful variant install:
- we acquire the same lockfile
- we remove our unique proc file
This should mean that procs can delete failed variant installs to new package dirs safely.
Note: There's always a chance that a rez proc will abort during the variant install for unexpected reasons. In this case, the unique proc file would stay present, and the package dir would never be cleaned up. However this would still be better than the current situation, where a failed new variant install always leaves the incomplete new package version dir behind.
Note: Currently failed variant installs do clean up their payload install (ie variant root path), but they do not clean up the hash_variant shortlink, if it is present. This is a separate but related bug.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at PackageRepository.pre_variant_install() and trace the existing .building-file and failed payload cleanup paths. Review how variant version directories, lockfiles, and hash_variant shortlinks are handled during concurrent installs. Done means failed installs remove incomplete new version directories safely without deleting data used by another process, while preserving successful installs and addressing the noted shortlink cleanup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100