nextflow-io / nextflow-io/nextflow
Support for confidential compute in GCP
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Hi,
Google Cloud recently released a feature to enable Confidential Computing. The goal is to "protect the confidentiality of data in the cloud by encrypting data-in-use while it’s being processed". It's a security practice they recommend and in my case, is enforced at the organization level. Since VMs scheduled through Nextflow don't use this feature by default, it causes my workflows to fail.
Currently, there is no way to set that up in the nextflow.config. @ejseqera proposed a nice workaround, which consists in using instance templates with confidential computing enabled. However it requires setting that up for every different type of compute we have (since, IIUC, cpu/memory/disk directives are not compatible with the instance template directive). I'm also not sure of
how that would play along with retry strategies that increase resources.
Finally, it also means managing all these instance templates along with resource requirement evolution, which is not ideal.
Would it be possible to include that feature as a GCP specific config? Something like:
gcp {
enableConfidentialComputing = true
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the GCP configuration in nextflow.config and trace how scheduled VMs use instance templates, resource directives, and retry strategies. Done means a GCP-specific confidential-computing setting can enable the feature without requiring separate instance templates for each resource configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, groovy
- Domain
- cloud, infrastructure, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100