TreeValGal status and issues

(from @fubar2)

# TreeValGal/EBPhib workflows: October 9 status

### 1\. Current state:

* Two workflows that could be combined into one.
* Need a new name
* Both running on EU and on VGP
* Basic tracks:
* [documentation draft](https://docs.google.com/document/d/1YbQZsFC8oDJIGos5yLFbApumpK4yIz02qfwYEypDfI8/edit)
* Current [WF on VGP](https://vgp.usegalaxy.org/u/fubar/w/treevalgalaugtwohaps-3)
* Dimers, coverage, SV:
* [documentation draft](https://docs.google.com/document/d/1kFCpCGt7Itb3R1mXiNSigd6_mT7-8J0mjYJppliPYtQ/edit#heading=h.i99j8o3qfj7)
* Current [WF on VGP](https://vgp.usegalaxy.org/u/fubar/w/dimer-density-sv-and-coverage-imported-from-url)
* New and with little exposure to curators or biologist target audience, so refinement from user feedback is needed to make it more useful.

Question: **Should these two WF JBrowse2 tracklists be**
**combined into one mongo \~50 track browser?** Jbrowse2 can cope if most are hidden on the track menu \- the unchecked boxes in the track menus at the side. Both currently use sniffles to make a SV track and two of the ***map one*** coverage tracks are also duplicated.

### 2\. Blockers

* Big shoutout to @bgruening @jennaj @natefoo @mvdbeek for helping make all the important ones on EU and VGP go away.
* ***Repeatmasker*** not stable on VGP. A [map-reduce solution](https://github.com/galaxyproject/usegalaxy-tools/issues/839#issuecomment-2394916446) proposed by @mvdbeek works for the reduce step, but the map repeatmasker over a collection step overwhelms something.
* ***Repeatmodeler*** \-\> TF models \-\> ***repeatmasker*** [may be broken too](https://matrix.to/#/!yShkfqncgjcRcRztBU:gitter.im/$ZyjObvIwbQZrRWwvjuB8rqZGzk-LouyH4AmqxtGirtI?via=gitter.im&via=matrix.org&via=integrations.ems.host). Ideally get both working if resources are available, but optional for now.

### 3\. What needs to be done for production?

* IWC submission being prepared for a PR (Ross and Bjoern)
* Process to gather inputs to run each new assembly:
* Two assembled haplotypes (same contig names);
* Optional NCBI gene/rna/protein taxon id and downloaded fasta files;
* Optional haplotype or reference fasta from two or more closely related species for sequence similarity mapping;
* Run repeatmodeler for repeatmasker if both can be made to work;
* SOP gathering all inputs, executing workflows, distributing JBrowse2 zip files to genomeArk and distributing Fediverse and every other possible notification. Bjoern mentioned some additional ideas for the SOP to minimise redundant work and maximise benefit.
* Minimise redundant VGP Galaxy storage demand as part of the SOP
* ? Libraries for the big files \- compressed and not. Lots of pointers.
* Outreach is key to get biologists to try them and provide comments.
* Push demonstrations to genomeArk assemblies
* Publicise the URI
* Make GTN tutorials on JBrowse2 and the WFs
* Need input to developer and user documentation linked above
* Need suggestions for improvements, bugs and feedback from curators

### 4\. Track lists for the two workflows \- potentially combined?

| WF1 | WF2 |
|-----|------|
|  |  |

Compare the two linked workflow drafts and the current VGP workflows, including their JBrowse2 track lists and Repeatmasker blockers. Start by determining whether the workflows should be combined, then define the production inputs, SOP, and documentation needed. Done would mean an agreed workflow structure and documented process for running and distributing results.

Written by the indexing model from the issue text.