con / con/git-annex

Simplify `Run datalad tests` step — `-c $toxini` no longer needed for datalad's pytest config

Open
#289 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29
Forks
5
PR merge metrics
No merged PRs in 30d

Description

## Summary

The `test-datalad` job across all four `build-*.yaml` workflows invokes
pytest with `-c "$toxini"` pointing at `datalad/tox.ini`. Historically
this was needed to pick up datalad's pytest markers, filterwarnings and
`python_files` pattern. Upstream datalad has since migrated all of that
into the [datalad pytest plugin][plugin] (`datalad/pytest_plugin.py`),
loaded via the `pytest11` entry point in `pyproject.toml`.

Confirmation from the current `datalad/tox.ini` (`maint`):

```ini
[pytest]
# Markers, filterwarnings, and python_files pattern are now registered
# via the datalad pytest plugin (datalad/pytest_plugin.py) which is
# loaded via the pytest11 entry point in pyproject.toml. This ensures
# consistent configuration for both datalad itself and all datalad
# extensions. Extension-specific pytest configuration can be added
# below if needed.
```

So `-c "$toxini"` currently provides no pytest configuration whatsoever.

[plugin]: https://github.com/datalad/datalad/blob/maint/datalad/pytest_plugin.py

## Why it's not zero-effort to just drop `-c "$toxini"`

Removing the flag has a side effect: pytest walks up from CWD looking
for a config file. Because non-Windows workflows currently run pytest
from `$PWD/__testhome__` (inside `$GITHUB_WORKSPACE`, our repo), pytest
would find **our own** `setup.cfg`'s `[tool:pytest]` block:

```ini
[tool:pytest]
# Put tests/ on sys.path so both the conftest and individual test
# modules can import shared helpers via `from _helpers import ...`
pythonpath = tests
```

That leaks our `tests/` directory onto `sys.path` during datalad's test
session. Probably harmless (no name collisions), but sloppy — the
datalad job has no business seeing our extra-tests helpers.

## Options (any of these resolves it)

1. **Run pytest from outside `$GITHUB_WORKSPACE`** on non-Windows too,
matching the Windows job's `$HOME/dl-testhome` pattern. Then drop
the `-c "$toxini"` flag and the `Checkout datalad` step for non-
Ubuntu (Ubuntu still needs it for `deploy_datalad-rootca`). Needs
verification that datalad's own tests are OK being run from a
working directory outside the repo.
2. **Replace `-c "$toxini"` with `-c /dev/null`** (portable via bash on
Windows too). Explicit "no config file, use the plugin's pyproject
entry-point registrations only". Least behavioural change.
3. **Keep `-c "$toxini"` as-is, update the comment** on the `Checkout
datalad` step (currently `# needed for ssh certs under ubuntu and
tox.ini everywhere`) to reflect the modern reason (pytest-config
isolation from our own `setup.cfg`).

Option 1 is the "right" long-term direction (fewer files to check out,
smaller build surface) but needs investigation — datalad tests may
implicitly depend on CWD being under the source tree. Options 2 and 3
are safer holding patterns.

## Files affected

- `.github/workflows/template/build-{{ostype}}.yaml.j2` (the Jinja
template — regenerate the 4 concrete `build-*.yaml` files after).
- `.github/workflows/build-ubuntu.yaml`
- `.github/workflows/build-macos.yaml`
- `.github/workflows/build-macos-arm64.yaml`
- `.github/workflows/build-windows.yaml`

## Related

- Windows-specific `$HOME/dl-testhome` CWD introduced in #287 /
8dcd089e24 (to avoid `ValueError: path is on mount 'D:', start on
mount 'C:'` when datalad code calls `os.path.relpath` across drives).
A "run from outside workspace" fix would generalize that pattern to
all platforms.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with .github/workflows/template/build-{{ostype}}.yaml.j2 and compare the four generated build-*.yaml workflows, focusing on the test-datalad job, its working directory, and the -c "$toxini" flag. Read the documented datalad pytest plugin and tox.ini context, then verify the chosen approach keeps datalad configuration isolated and the generated workflows remain consistent. Done means the unnecessary configuration handling is addressed and the datalad CI job still runs correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.