snakemake / snakemake/snakemake

Have linting to detect `pathvar` anti-patterns

Open
#4,149 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
2.9k
Forks
656
Avg merge
4d 12h
Merged PRs (30d)
9

Description

(original issue: https://github.com/snakemake/snakefmt/issues/283)

Is your feature request related to a problem? Please describe.
Using pathvars might introduce issues if devs are not pristine in how they add them to rules.
For example, if you forget to add <resources>/... to just one rule input, you'll end up with a file placed outside of wherever you re-route <resources> to.

This is often hard to track in large enough workflows.

Describe the solution you'd like
It'd be nice to have a linting option that tests that all rules in a workflow make use of pathvars, at least for <resources>, <logs>, and <results>.
Additionally, it'd also be useful to check that custom pathvars also respect default pathvars to avoid anti-patterns

Suggested checks:

  1. all rule logs, inputs, outputs and benchmarks start with their equivalent <> default (e.g., inputs start with <resources>) or a custom alternative (e.g., an input could be <user_shapes>).
  2. Custom pathvar defaults respect generic pathvars via nesting (so that if they are unspecified, their default respects them)
pathvars:
    # will still respect the 'resources' pathvar if not specified
    user_shapes="<resources>/user/{shapes}/shapes.parquet",

Describe alternatives you've considered
Lots of visual inspection, I suppose?

Additional context
None

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by locating Snakemake's existing linting entry points and the pathvars parsing or validation code, then determine where workflow rules and custom pathvar defaults are represented. Done means linting reports the listed pathvar anti-patterns across rule inputs, outputs, logs, benchmarks, and nested custom defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.