VEuPathDB / VEuPathDB/ApiCommonWorkflow
Make genomics project dependency lists data-driven instead of hand-copied
Nobody has claimed this yet.
- Dominant language
- Perl
- Stars
- 1
- Forks
- 2
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 11
Description
Problem
The list of genomic projects (PlasmoDB, ToxoDB, TriTrypDB, VectorBase, AmoebaDB, CryptoDB, FungiDB, PiroplasmaDB, GiardiaDB, MicrosporidiaDB, TrichDB, HostDB, and now HelminthDB) is hand-copied into 7 separate blocks across 2 files:
Main/lib/xml/workflowTemplates/genomicsRoot.xml — 5 blocks:
sampleDetailsForEdaOrthoMCLDependentalphaFoldExternalResourceUrlanalyzeWebreadyPartitionedTables
Main/lib/xml/workflow/orthomclGenomics.xml — 2 blocks:
checkOrthoCoreProteomeDirsdependsExternallistcheckOrthoPeripheralProteomeDirsdependsExternallist
The only thing keeping them in sync is a comment at genomicsRoot.xml:59:
ATTENTION: This graph has dependsExternals by PROJECT. If you add a project above, you must also add it here
A load-bearing sticky note is the tell that the structure is wrong. Adding HelminthDB (#106) meant 7 edits and a manual audit to confirm nothing was missed; a silently omitted entry produces a workflow graph that runs but quietly skips a project's data.
Proposed direction
The project list is data, not structure. Options, roughly in order of preference:
-
datasetTemplate-driven dependency expansion — thegenomicsProjectdatasetTemplatealready enumerates projects to instantiate the per-project subgraphs. Extend ReFlow so a step can declare a dependency on all instances of a template (e.g.<dependsAll template="genomicsProject"/>, and fororthomclGenomics.xmlsomething like<dependsExternalAll template="genomicsProject" step="checkOrthoCoreProteomeDirs"/>). Single source of truth, no duplication anywhere. -
Shared constant + expansion — declare the list once (
<constant name="allGenomicsProjects">) and have the XML expand it intodepends/dependsExternalentries. Less invasive than (1), still one place to edit. -
Validation-only fallback — if neither expansion mechanism is worth building, add a test/lint step that asserts all 7 blocks contain the same project set. Doesn't remove the duplication but converts a silent skip into a loud failure.
Option 1 also makes the naming convention (${projectName}_checkOrthoCoreProteomeDirs) explicit in the graph rather than implied by string concatenation in two places.
Acceptance
Adding project #14 should be a one-line change in one file, with no cross-file audit required.
🤖 Generated with Claude Code
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the seven dependency blocks in Main/lib/xml/workflowTemplates/genomicsRoot.xml and Main/lib/xml/workflow/orthomclGenomics.xml, then trace how ReFlow expands datasetTemplate dependencies. Compare the proposed template-driven and constant-based approaches, and inspect existing workflow tests or validation commands if present. Done means adding project #14 requires one line in one file without a cross-file audit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- perl, xml
- Domain
- build-system, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100