spack / spack/spack

Inconsistent behaviors of spec inspection

Open
#37,142 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.1k
Forks
2.5k
Avg merge
3d 4h
Merged PRs (30d)
82

Description

I am not sure whether this is a bug or an intended feature, and appreciate some clarifications on Spack 0.19.2

First, I built Open MPI with the following command:

$spack install openmpi@4.1.4 +internal-hwloc +legacylaunchers +pmi fabrics=auto schedulers=slurm arch=zen3 %gcc@11.3.0 ^pkgconf %gcc@4.8.5 ^numactl %gcc@4.8.5 ^openssh %gcc@4.8.5

The trailing dependencies with gcc@4.8.5 is to stop Spack from rebuilding them using gcc@11.3.0, which is redundant.

Then, performing inspection with netlib-scalapack gave the following specs:

 $ spack spec -I netlib-scalapack@2.2.0 arch=zen3 %gcc@11.3.0
...
 -       ^cmake@3.24.3%gcc@11.3.0~doc+ncurses+ownlibs~qt build_system=generic build_type=Release arch=linux-centos7-zen3
 -           ^ncurses@6.3%gcc@11.3.0~symlinks+termlib abi=none build_system=autotools arch=linux-centos7-zen3
 -           ^openssl@1.1.1s%gcc@11.3.0~docs~shared build_system=generic certs=mozilla arch=linux-centos7-zen3
 -               ^ca-certificates-mozilla@2022-10-11%gcc@11.3.0 build_system=generic arch=linux-centos7-zen3
 -       ^openmpi@4.1.4%gcc@11.3.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=none schedulers=none arch=linux-centos7-zen3
 ...

Here, Spack wanted to rebuild Open MPI again since required specs were different from the above one.

But what confused us was the following command would correctly re-use the previously built Open MPI:

$ spack spec -I netlib-scalapack@2.2.0 arch=zen3 %gcc@11.3.0 ^cmake %gcc@4.8.5 ^openmpi
...
[+]      ^cmake@3.24.3%gcc@4.8.5~doc+ncurses+ownlibs~qt build_system=generic build_type=Release arch=linux-centos7-x86_64_v3
[+]          ^ncurses@6.3%gcc@4.8.5~symlinks+termlib abi=none build_system=autotools arch=linux-centos7-x86_64_v3
[+]          ^openssl@1.1.1s%gcc@4.8.5~docs~shared build_system=generic certs=mozilla arch=linux-centos7-x86_64_v3
[+]              ^ca-certificates-mozilla@2022-10-11%gcc@4.8.5 build_system=generic arch=linux-centos7-x86_64_v3
[+]      ^netlib-lapack@3.10.1%gcc@11.3.0~external-blas~ipo+lapacke+shared~xblas build_system=cmake build_type=RelWithDebInfo arch=linux-centos7-zen3
[+]      ^openmpi@4.1.4%gcc@11.3.0~atomics~cuda~cxx~cxx_exceptions~gpfs+internal-hwloc~java+legacylaunchers~lustre~memchecker+pmi+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=auto schedulers=slurm arch=linux-centos7-zen3

cmake is first among the dependency chain. By forcing ^cmake @4.8.5, Spack will automatically pick up dependencies which are already built either versions of gcc compiler. Is this by design ?

Thanks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing both spack spec -I commands from the issue on Spack 0.19.2 and compare their concretized dependency graphs, especially the forced compiler on cmake and the selected openmpi variant. Trace the dependency-resolution entry point that handles these constraints; done means the behavior is explained and either corrected with coverage or documented as intentional.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.