AcademySoftwareFoundation / AcademySoftwareFoundation/rez

Supporting Multiple Compilers

Open
#42 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
369
Avg merge
12d 3h
Merged PRs (30d)
5

Description

We are starting to build packages with both ICC and GCC variants. As ICC requires GCC at run time (and I think build time), my `icc` package looks something like this:

```
name: icc
version: 14.0.0
variants:
- [ CentOS-6.2, gcc-4.1.2 ]
- [ CentOS-6.2, gcc-4.4.6 ]
commands:
- CXX=!ROOT!/bin/icpc
- CC=!ROOT!/bin/icc
```

To build all ICC and GCC variants of a higher level package, we must do something like this:

```
name: foo
version: 1.0.0
variants:
- [ CentOS-6.2, gcc-4.1.2 ]
- [ CentOS-6.2, gcc-4.4.6 ]
- [ CentOS-6.2, icc-14.0.0, gcc-4.1.2 ]
- [ CentOS-6.2, icc-14.0.0, gcc-4.4.6 ]
```

This creates three problems:
1. The `icc` package redefines the CXX and CC environment variables, which are also defined in the `gcc` package. Rez does not allow this (it's a conflict) and so temporarily I've commented this restriction out. I understand the restriction, but maybe we can loosen it a little?
2. The `foo` package now has unbalanced variants. This doesn't appear to be a problem (it works) but is maybe not considered very neat. This would perhaps benefit from naming the variant folders as we discussed in [another thread](https://github.com/nerdvegas/rez/issues/21#issuecomment-19083476).
3. In theory we are able to successfully mix and match compilers are runtime - they are not mutually exclusive. Assuming another package `bah` with the same variants as `foo` we can build an environment using the `gcc-4.4.6` variant of `foo` and the `icc-14.0.0, gcc-4.4.6` variant of `bah`.

Currently (I believe) a call `rez-env foo bah` will pick the first variant that creates a resolution (both GCC), `rez-env foo bah icc-14.0.0 gcc-4.4.6` will resolve for both packages to use the ICC variant. Is there currently some syntax I can use to be more specific, e.g. `rez-env foo:gcc-4.4.6 bah:icc-14.0.0`?

Perhaps this is an extension of what was discussed in [issue 21](https://github.com/nerdvegas/rez/issues/21).

To extend point 1) a little further with another use case:

We want to define a `PYTHON_EXE` environment variable which points to the current python interpreter for the resolved environment. For example:

```
name: maya
requires:
-python-2.6
commands:
- export PYTHON_EXE=mayapy

name: python
version: 2.6.6
commands:
- export PYTHON_EXE=python2.6
```

As both the `python` and `maya` package define `PYTHON_EXE` we get a conflict. The way around this I see would be to define the `python-2.6` requirement of `maya` as a separate `mayapy` package.

```
name: maya
requires:
-mayapy-2.6

name: mayapy
version: 2.6.6
commands:
- export PYTHON_EXE=mayapy
```

But our python applications have three variants for python 2.5, 2.6 and 2.7 and for applications which are pure python, they work in a normal python interpreter, as well as Maya/Nuke/Houdini's. To follow this approach we'd have to build many more variants of essentially the same code which seems wasteful:

```
name: foo
variants:
- [ python-2.5 ]
- [ python-2.6 ]
- [ python-2.7 ]
- [ mayapy-2.6 ]
- [ hython-2.6 ]
- [ nukepy-2.6 ]
```

I don't like this approach. Any suggestions?

Contributor guide

Open the contributing guide

Research direction

Start with the rez-env behavior described in the issue and review the linked issue 21 discussion. Determine how compiler and interpreter variants should coexist, how conflicting environment variables should be handled, and whether package-specific variant selection syntax is needed. Done should include an agreed design for these cases and corresponding implementation and validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.