galaxyproject / galaxyproject/galaxy

Things to do to make the job cache more useful

Open
#6,887 11 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/job-caching kind/enhancement status/planning
Dominant language
Python
Stars
1.9k
Forks
1.2k
Avg merge
2d 7h
Merged PRs (30d)
167

Description

  • Datasets imported from a library are not recognized as being the same
  • The input and output datasets need to be owned by the user -- accessible should be enough (though that might be a concern with our accessible by default policy)
  • #6781 having the HID as part of the name really requires one to have the exact same history structure
  • A tool property that says "ignore the element_identifier / name when caching" would be helpful
  • A tool property to indicate the tool outputs are not reproducible (e.g. tools that produce random results on purpose)
  • An additional setting that would allow the user to specify that an exact metadata match isn't necessary
  • Enable cache override via UI element in the tool form.
  • A preview
  • Make datasets read-only in object store https://github.com/galaxyproject/galaxy/issues/8263
  • If we have a hash for dataset contents we can "resume" cached execution when intermediate files get deleted or when the dataset metadata for an input file didn't match but the metadata mismatch didn't result in a different output
  • Add something to the UI to indicate that a job was recovered from cached data
  • https://github.com/galaxyproject/galaxy/issues/20050

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reviewing the unchecked items and linked issues #6781, #8263, and #20050, then narrow the work to one agreed improvement with explicit completion criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.