nextflow-io / nextflow-io/nextflow

Custom AWS Batch job names

Open
#1,232 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

executor/aws-batch good first issue stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Usage scenario

Hello! We have several people using Nextflow on AWS and it's getting confusing whose jobs are whose in the job queue. I'd like to be able to scan the job listing and see whose jobs are where, and since I often submit multiple workflows, also the workflow.runName to keep track of which workflow this e.g. trimming step is from. For example, Instead of "make_additional_gtf_1" I'd like the job name to be: maniac_franklin__olgabot__make_additional_gtf_1

Suggest implementation

One implementation I can think of is to add a variably-defined name_prefix to the workflow params:

params {
    name_prefix = "${workflow.runName}__${workflow.userName}"
} 

(Highlight the main building blocks of a possible implementation and/or related components)

Hello,
I'd like to add the Nextflow variable ${workflow.runName} and local username $USER to the AWS batch job name

Is this possible to redefine in a configuration? I see that one can label individual process but I'd like to do this for all processes.
Thank you!
Warmest,
Olga

Originally asked: https://groups.google.com/forum/#!topic/nextflow/odWQRzwtgAI

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how AWS Batch job names are generated and how workflow.runName, workflow.userName, process labels, and configuration are exposed. Done should mean users can configure a prefix applied to all AWS Batch jobs and distinguish both the workflow and submitting user in the job listing.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, groovy
Domain
cloud, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.