AcademySoftwareFoundation / AcademySoftwareFoundation/rez
Implementing multi-version packages
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 369
- Avg merge
- 12d 3h
- Merged PRs (30d)
- 5
Description
Hello (@nerdvegas) Allan,
First of all thank you very much for all your work on rez, it will really help us streamline our tool deployment.
My name is Ira and and I am closely working with (@instinct-vfx) Thorsten to incorporate rez into our studio's workflow. I will be helping developing and extending rez as required to match our needs as such I wanted to introduce myself. A lot of the changes that we need to implement will possibly benefit the whole community therefore I look forward to contributing to the project as a whole.
My first task is something that you and Thorsten have already discussed about at the forums on this [post](https://groups.google.com/forum/#!topic/rez-config/R1iJ9iE8Txg) about the future of rez-bind.
I have read the post carefully and slowly slowly started implementing the feature while I get myself accustomed to the code.
Based on your statement below:
>> I think it makes sense to add a new optional `versions` attribute to the package.py. If provided by the package you're attempting to build, you would need to specify the package that you want explicitly.
>>In other words, you could have a single package.py for 3dsmax, and just install it with `rez-build --install --version=1.2.3`.
I understood how I should be approaching this task but as you can imagine a lot of questions popped up as such I would like to gather as much information as possible in order to make sure that we are in line on how this should be implemented before opening a pull request.
Based on my understanding since the versions attribute is to become part of core rez rather than a user custom attribute it needs to be added to the schema to ensure validity of its content correct?
1. Schema modifications
* package_maker__.py (package schema)
* package_serialize.py (package key order, package serialize schema)
* package_resources_.py (package schema, package pod schema
I am also wondering about the type of the attribute, for example I can see that version has the Use(Version) requirement so I was originally thinking to use [basetring] along with Version for the versions attribute but after further consideration a simple [basestring] seems to be enough since that string will eventually get assigned to a version attribute which will in turn get validated.
What do you think? Also are there any more places that the schema needs to be modified if at all?
Next I wanted to ask about adding the flag to the rez-build, (also rez-release) commands for selecting a particular version.
2. Add command line flag to rez-build, rez-release to enable version selection
* build.py (add argument to argparse)
* name it something like build version to avoid user confusion version might be rez-build version or rez version etc
One approach will of course be to add an argument to the build.py that can be passed along with install to select a version. A few things I am thinking that might be needed are:
- ensure that either version or versions is defined in the package before processing
- a multi-version package should only include versions and the installed package will be having the version assigned to the one specified during install
- versions string validation? - this is related to the schema above
- ensure that the specified version to install is actually in the versions list
An issue I am facing is that although adding an extra argument to the build command is not a problem, passing that argument all the way to where it needs to get parsed is messy. It seems that I need to keep passing the argument to the get_developer_package and get_current_developer_package which in turn will pass it to DeveloperPackage.from_path then to load_from_file until it finally gets parsed in the load.py (package.py file).
Not only this is messy but there are also timing issues where setting the common parser arguments calls the get developer package before the passed version actually gets parsed. I noticed that during the setup_parser the get developer package is not being called.
```
def setup_parser_common(parser):
"""Parser setup common to both rez-build and rez-release."""
from rez.build_process_ import get_build_process_types
from rez.build_system import get_valid_build_systems
process_types = get_build_process_types()
parser.add_argument(
"--process",
type=str,
choices=process_types,
default="local",
help="the build process to use (default: %(default)s).")
# add build system choices valid for this package
#args = parser.parse_args()
#package = get_current_developer_package(build_version=args.build_version)
package = get_current_developer_package(build_version="2017.0.0")
clss = get_valid_build_systems(os.getcwd(), package=package)
```
Above I am simply hard-coding a version for testing but it shows when the timing issue occurs.
The way I am actually using the build version flag is to set an argument value in an early bind which I realized can take a single argument. Once the build version is passed to the file I use the
_process function (process_python_objects) in serialize.py to set the version to the passed argument.
```
def process_python_objects(data, filepath=None, build_version=None):
.....
```
You can see how deep I need to pass the build version flag...
```
....
if args:
# this 'data' arg support isn't needed anymore, but I'm
# supporting it til I know nobody is using it...
#
value_ = fn(build_version) # use build version to set version attribute
else:
value_ = fn()
...
```
I could not find any documentation regarding the use of arguments to early binding attributes, how are we supposed to use them?
Here is my current setup for a multi-version package.py file;
```
name = "3dsmax"
authors = [
"Ira"
]
uuid = "examples.3dsmax_py"
versions = ["2017.0.0", "2018.0.0"]
requires = [
"python"
]
@early()
def version(build_version): # this get set in the process function as mentioned above
return build_version
@early()
def description(build_version):
return "%s version %s" % (this.name, build_version)
def commands():
env.PYTHONPATH.append("{root}/max")
env.PATH.append("{root}/bin")
```
So what I would like to know is what would be the best approach to pass that build version to the location it needs to get parsed and assigned as a version attribute avoiding timing issues or deep argument passing?
Also what is the recommended way of using arguments for early binds? Is it meant to be used only within the context of package.py?
Another completely different approach to the one described above would be to possibly use the [passing arguments](https://github.com/nerdvegas/rez/wiki/Building-Packages#passing-arguments) method. Is this feasible? Add a parse_build_args.py source file to include the extra build version argument and then access it somehow in the package.py to set the version attribute? This sounds like a much cleaner easier approach that does not even touch rez core.
I am sure there are still more things to discuss but this is already becoming too long so I will stop here for now. I look forward to your feedback and advice and I will be more than happy to talk over the details at your earliest possible convenience. If it makes sense we can even arrange a call or something so please let me know.
I am maintaining a [fork](https://github.com/lambdaclan/rez) of rez under my user name for testing purposes and when the time comes I will be opening a pull request please feel free to contact me in either repo.
Thank you very much.
Edit: Just realized there is a similar old issue at #54 so I will look into that as well in case it contains something of value.
Contributor guide
Assessment
This issue has not been assessed yet.