spack / spack/spack

test suite doesn't pass under Docker

Open
#8,518 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

containers docker tests
Dominant language
Python
Stars
5.1k
Forks
2.5k
Avg merge
3d 4h
Merged PRs (30d)
82

Description

The test suite does not pass under Docker, during Dockerfile build, with errors about locking.

Thanks for your hard work on Spack. Please let me know any additional clarification I can provide.

Expected Result

Test suite passes.

Actual Result

The RUN spack test instruction fails with the following errors:

=================================== FAILURES ===================================
__________ test_upgrade_read_to_write_fails_with_readonly_file[/tmp] ___________

private_lock_path = '/tmp/tmpO2kFFA/lockfile'

    def test_upgrade_read_to_write_fails_with_readonly_file(private_lock_path):
        # ensure lock file exists the first time, so we open it read-only
        # to begin wtih.
        touch(private_lock_path)

        with read_only(private_lock_path):
            lock = Lock(private_lock_path)
            assert lock._reads == 0
            assert lock._writes == 0

            lock.acquire_read()
            assert lock._reads == 1
            assert lock._writes == 0
>           assert lock._file.mode == 'r'
E           AssertionError: assert 'r+' == 'r'
E             - r+
E             + r

lib/spack/spack/test/llnl/util/lock.py:597: AssertionError
____________________________ test_lock_checks_user _____________________________

tmpdir = local('/tmp/pytest-of-root/pytest-0/test_lock_checks_user0')

    def test_lock_checks_user(tmpdir):
        """Ensure lock checks work."""
        path = str(tmpdir)
        uid = os.getuid()

        # self-owned, own group
        tmpdir.chown(uid, uid)

        # safe
        tmpdir.chmod(0o744)
        spack.util.lock.check_lock_safety(path)

        # safe
        tmpdir.chmod(0o774)
        spack.util.lock.check_lock_safety(path)

        # unsafe
        tmpdir.chmod(0o777)
        with pytest.raises(spack.error.SpackError):
            spack.util.lock.check_lock_safety(path)

        # safe
        tmpdir.chmod(0o474)
        spack.util.lock.check_lock_safety(path)

        # safe
        tmpdir.chmod(0o477)
>       spack.util.lock.check_lock_safety(path)

lib/spack/spack/test/llnl/util/lock.py:965:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = '/tmp/pytest-of-root/pytest-0/test_lock_checks_user0'

    def check_lock_safety(path):
        """Do some extra checks to ensure disabling locks is safe.

        This will raise an error if ``path`` can is group- or world-writable
        AND the current user can write to the directory (i.e., if this user
        AND others could write to the path).

        This is intended to run on the Spack prefix, but can be run on any
        path for testing.
        """
        if os.access(path, os.W_OK):
            stat_result = os.stat(path)
            uid, gid = stat_result.st_uid, stat_result.st_gid
            mode = stat_result[stat.ST_MODE]

            writable = None
            if (mode & stat.S_IWGRP) and (uid != gid):
                # spack is group-writeable and the group is not the owner
                writable = 'group'
            elif (mode & stat.S_IWOTH):
                # spack is world-writeable
                writable = 'world'

            if writable:
                msg = "Refusing to disable locks: spack is {0}-writable.".format(
                    writable)
                long_msg = (
                    "Running a shared spack without locks is unsafe. You must "
                    "restrict permissions on {0} or enable locks.").format(path)
>               raise spack.error.SpackError(msg, long_msg)
E               SpackError: Refusing to disable locks: spack is world-writable.
E                   Running a shared spack without locks is unsafe. You must restrict permissions on /tmp/pytest-of-root/pytest-0/test_lock_checks_user0 or enable locks.

lib/spack/spack/util/lock.py:91: SpackError
======== 2 failed, 1037 passed, 90 skipped, 1 xfailed in 415.11 seconds ========
The command '/bin/sh -c spack test' returned a non-zero code: 1

RUN spack --enable-locks test gives 35 failures instead of 2.

Steps to reproduce the issue

Use docker build on the following Dockerfile:

FROM debian:stretch

ENV DEBIAN_FRONTEND noninteractive

RUN    apt-get update \
    && apt-get install -y apt-utils

# Spack needs curl, git, make, and unzip to install.
# The other packages are needed for Spack unit tests.
RUN apt-get install -y \
    curl \
    g++ \
    git \
    make \
    patch \
    procps \
    python \
    python-pkg-resources \
    unzip

# Install Spack.
ENV SPACK_REPO https://github.com/spack/spack
ENV SPACK_VERSION 818e748
RUN git clone $SPACK_REPO
RUN cd spack && git checkout $SPACK_VERSION

# Set up environment to use Spack. (We can't use setup-env.sh because the
# Dockerfile shell is sh, not Bash.)
ENV PATH /spack/bin:$PATH
RUN spack compilers

# Test: Install a small package.
RUN spack install libsigsegv

# Test: Run Spack test suite.
RUN spack test

# Clean up.
RUN spack clean --all
Information on your system
  • Debian Stretch.
  • Docker version 17.12.0-ce, build c97c6d6
  • Spack 0.11.2 or commit 818e748 (tip of develop as of right now).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the provided Debian Dockerfile and docker build. Inspect lib/spack/spack/test/llnl/util/lock.py alongside lib/spack/spack/util/lock.py, focusing on the reported lock mode and permission checks. Done means the Docker build's spack test command passes without the two reported failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.