AcademySoftwareFoundation / AcademySoftwareFoundation/rez

Multi Architecture Support

Open
#154 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.1k
Forks
369
Avg merge
12d 3h
Merged PRs (30d)
5

Description

We have a need to build packages for multiple cpu architectures. For example, a library will be built three times - once for AVX2, once for AVX and once for SSE2. We are looking to model this in Rez and I would appreciate some feedback.

The three 'variants' of the package can all be built on the same host with different compiler flags (so a distributed build is not required).

At runtime there is a fallback hierarchy of support available - if a the runtime host has AVX2 available, but the package only has AVX this is ok. This makes it different to the normal package/variant structure where packages are mutually exclusive.

Currently, we have two proposals:
1. In the `package.yaml` add a new field. This will be a list of architectures that package supports. It would be validated against a master list of architectures from `rezconfig`. At build time, we have custom CMake macros for `add_library` and `add_executable` which automatically creates targets based on the content of this list. At runtime, rez makes both the current host's architecture available (`{system.architecture}`) as well as the highest level of support for the current package (`{supported_architecture}`). This allows the `commands` to then set `LD_LIBRARY_PATH` correctly.
2. model the architectures as normal variants (which aren't mutually exclusive). This means the build only needs to detect the architecture for the current variant to set the correct compiler flags. At runtime, Rez adds implicit packages for each supported architecture on the current host machine. The variant sorting (Fede's pull request) then ensures the correct variant of each package is loaded.

Option 2. is closer to what rez currently supports in terms of variants etc so would require the smallest change to rez. However this feels susceptible to changes in the resolution algorithm which might not work in our favour. Option 1 is a larger change to rez but feels a bit more concrete and controllable.

I would be interested in your thoughts. Currently we are siding with Option 2. (after a long time being set on Options 1.).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.