DIRACGrid / DIRACGrid/intercede

[Feature]: HTCondorCEBackend — grid CE over the shared `_htcondor` core

Open
#23 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3
Forks
3
PR merge metrics
No merged PRs in 30d

Description

## User Story

As the WMS consumer,
I want an `HTCondorCEBackend` submitting to an HTCondor-CE with IDTOKENS and spooled file
transfer,
So that the second major grid-CE path exists on the new contract, including the destructive-fetch
semantics that motivated IC-ADR-002.

## Feature Description

- **`HTCondorCEBackend`**: `JobBackend` + `OutputRetriever` + `Cancellable`.
Deliberately **no `LoadReporter`** (DIRAC's `getCEStatus` is an explicit "not supported") and no
live diagnostics — capability absence is the point, and the integration suite must show the
narrowing works.
- Reuses the **`_htcondor` core** (submit description, ClassAd parsing, state
map); differs in transport/auth: remote schedd (`-pool`/`-remote`) or CE submission, grid
universe routing, `-spool`.
- **Token + proxy together** per IC-ADR-003 (its Motivation 7 / all-of `kinds`): SCITOKENS/IDTOKENS
channel auth (token file + `_CONDOR_*` env injection, audience `:9619`) **plus** the X.509
proxy materialised into the submit description (`use_x509userproxy = true`) — consumed
*site-side* (VOMS attributes → APEL accounting), including the
`_CONDOR_DELEGATE_JOB_GSI_CREDENTIALS=false` workaround (HTCONDOR-2904). The pure-IDTOKENS
`basic` stack config exercises the token path; a proxy-bearing config is the ADR-002 auth axis.
- **Destructive retrieval**: `condor_transfer_data` then declared `destructive=True`
(HTCondor ≥ 25.8: completed spooled job leaves the queue); evaluate `leave_in_queue` as the
re-fetchability override and record the decision. Coarse bounded materialisation (submit-time
`transfer_output_files` allow-list, transfer timeout, post-hoc size/count check).
- **Phase 2 wiring**: contract suite against the `htcondor` stack's CE path (markers
`remote and htcondor`); the **fetch-twice `destructive_fetch` test** is the flagship regression
(versions axis `[lts, latest]` makes 25.8-class changes red on the Renovate PR).

## Definition of Done

- [ ] Contract suite green against the `htcondor` stack CE path with IDTOKENS
- [ ] Fetch-twice test asserts the declared one-shot semantics; `leave_in_queue` decision recorded
- [ ] `isinstance(be, LoadReporter)` is False and the suite's capability-skip path exercises it
- [ ] Kill path green; per-id outcomes on partial batches

## Alternatives Considered

- Expressing it as `BatchBackend(HTCondorCETransport, HTCondorScheduler)` — open ADR question;
blocked on a transport-level "destructive get" notion. Start monolithic, revisit (share only
`_htcondor` functions, never a subtype).
- GSI/VOMS auth — out of scope; IDTOKENS-first per ADR-002 §4.

## Related Issues

Blocked by: # (`_htcondor`), #. Validated by: #5 / htcondor stack.

## Additional Context

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.