snakemake / snakemake/snakemake-interface-common

Snakemake errors out if plugin is not installed

Open
#94 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
7
Avg merge
31m
Merged PRs (30d)
1

Description

This is effectively the same as https://github.com/snakemake/snakemake-storage-plugin-fs/issues/40, but should likely be handled by snakemake interface rather than the plugin itself.

In the use case, I have a large file in my working directory that benefits from being copied to node-local storage on an HPC (i.e. a job-defined $TMPDIR). I can do this with the strorage-plugin-fs, setting remote-job-local-storage-prefix: "$TMPDIR" in my profile config. The rule looks like

rule test:
     input:
         BAM=storage.fs("genome.bam")
     output:
          ...

If I run this without the plugin being installed, I get an error like

Error loading Snakemake plugin fs: The package snakemake-storage-plugin-fs is not installed.

coming from here.

Given the file exists, it would be ideal for the fallback to just ignore the storage.fs "decoration" and use the original file in the working directory, maybe emitting a warning. This may be complicated by remote storage plugins where the file only exists remotely and so needs to be handled by the plugin, but at least for the fs plugin this shouldn't be an issue.

Maybe deciding if a plugin is optionally helpful versus necessary is challenging, but the main snakemake process erroring out because a plugin is not installed is very limiting.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the plugin-loading exception at snakemake_interface_common/exceptions.py line 79 and trace how a missing storage plugin is handled for the fs example. Check how local file existence and remote-plugin requirements can be distinguished; done means an existing local file can proceed without the plugin, with an appropriate warning, while required remote handling still errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.