nextflow-io / nextflow-io/nextflow

Support specifying disk image for additional disk with Google Batch (e.g. Image with reference files)

Open
#4,534 1 comment 0 reactions 0 assignees View on GitHub

@hnawar is already working on this.

Since Nov 22, 2023.

  • #4535 by @hnawar — open
executor/google-batch stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

New feature

Support specifying a disk image for additional disk when using Google Batch. This would be useful in many cases but particularly with large reference files which can be both time consuming and expensive.
Using DIsk image is faster and cheaper as the Disk is created from the image before the provisioning the VM

Usage scenario

User creates a Disk image containing all reference files
User specifies the image as part of the disk directive in the process

Suggest implementation

Update Google Batch Task handler to use disk image if both type and image are specified. This will take advantage of the current code which applies to disk type

This could look like this

I've created a sample implementation in a PR

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Google Batch Task handler and the sample implementation referenced by the issue. Check how disk directives currently handle disk type, then verify that specifying both a disk type and image uses the image for the additional disk; the linked pull request indicates work is already underway.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, groovy
Domain
cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.