DIRACGrid / DIRACGrid/intercede

[Feature]: Adopt the CE-migration ADRs and build the integration-test scaffold (Phase1)

Open
#9 1 comment 0 reactions 1 assignee Claimed by @aldbr View on GitHub
Dominant language
Python
Stars
3
Forks
3
PR merge metrics
No merged PRs in 30d

Description

## User Story

As an interCEde maintainer and contributor,
I want the migration ADRs merged and a shared integration-test scaffold in place,
So that the in-flight backend PRs (#6, #7, #8) harmonize onto one agreed layout instead of
diverging, and every later stack has a template to slot into.

## Feature Description

This issue tracks the two foundational PRs and links both:

1. **The ADR PR — #4.** Adds `docs/adr/` with **IC-ADR-001** (Computing Element
interfaces & the DIRAC → interCEde migration), **IC-ADR-002** (Integration testing against
containerized backends) and **IC-ADR-003** (backend credential model — typed credentials,
declared requirements, provider-supplied), plus the template and index.

2. **The scaffold PR (new, branched off #4 ).** Lands the `tests/integration/` skeleton
that IC-ADR-002 describes — the *target* the three in-flight backend PRs converge onto. Stacked
on the ADR branch (so its diff is scaffold-only); retargeted to `main` once #4 merges.

Scaffold contents (Phase 0 — native-binary friendly, no dependency on the not-yet-existing
`intercede` package):
- `tests/integration/stacks.yml` — manifest seeded with the three stacks' entries so each author
sees where their line goes;
- `tests/integration/README.md` — the **harmonization guide / stack contract + checklist**;
- `tests/integration/stacks/_template/` — `compose.yml` skeleton (network, `credentials` volume,
`HEALTHCHECK` block) + `config/basic/intercede-client.toml` template;
- `tests/integration/stacks/_images/` — Dockerfile conventions (`ARG` + `# renovate:` annotation);
- `tests/integration/conftest.py` — thin `--stack`/`--config` options + capability-skip helper;
- integration-test **markers registered in `pyproject.toml`**
(`remote scheduler arc htcondor slurm ssh destructive_fetch`);
- a **skeleton `.github/workflows/integration.yml`** that reads `stacks.yml`, so authors can
delete their bespoke per-PR workflows against a real target.

Once this lands, harmonizing a stack = *move files under `stacks//` + add a manifest entry +
follow the checklist* — not "invent structure" or "rewrite against interCEde".

## Definition of Done

- [ ] PR #4 merged — `docs/adr/{IC-ADR-001,IC-ADR-002,IC-ADR-003,IC-ADR-XXX_template,index}` on `main`
- [ ] Scaffold PR merged (based on `adr_migration`, retargeted to `main` after #4)
- [ ] `tests/integration/` skeleton in place: `stacks.yml` (3 seeded stacks), `README.md`
checklist, `stacks/_template/`, `stacks/_images/` conventions, `conftest.py`, markers in
`pyproject.toml`, skeleton `integration.yml`
- [ ] A stack is runnable locally with the documented two-liner
(`docker compose … up --wait` + `pixi run pytest … --stack=`)
- [ ] Harmonization guide linked from #5, #, # so all three authors
work from the same target
- [ ] Follow-up issues filed for the deferred consolidation: manifest-driven CI matrix (fleshing
out `integration.yml`), prebuilt GHCR images, Renovate two-lane + version window, and the
`arc-condor` / `ssh-condor` stacks

## Alternatives Considered

- **Merge the three prototypes as-is, consolidate later.** Rejected: without a shared target they
re-diverge on the exact axes they already differ on (systemd vs foreground, Podman vs Docker,
Ubuntu vs EL9), making the later consolidation larger than doing the scaffold first.
- **Fold the scaffold into the ADR PR (#4).** Rejected: keeps the ADR reviewable as pure docs and
lets the scaffold evolve without re-opening the ADR; the stacked-PR link keeps them associated.
- **Full protocol-based contract suite now.** Deferred — the `intercede` Protocols
(IC-ADR-001) don't exist yet, so Phase 0 tests use native client tools.

## Related Issues

Implemented by: #4 (ADR docs) and the new scaffold PR
Related to: #5 (htcondor stack), # (PR #7), # (PR #8) — all three
harmonize onto this scaffold
Blocks: #5, #, # (they need the scaffold as their target)

## Additional Context

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.