nextflow-io / nextflow-io/nextflow
can't write output index files using google-batch executor & gs storage bucket for output dir
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Bug report
Expected behavior and actual behavior
the index file created within the output scope fails (java.lang.UnsupportedOperationException: APPEND) when run on google batch using a gs storage bucket as the output dir. it works locally and on aws batch and if i either set enabled = false or remove the index scope from the output scope. i tried to use a local path for the index (e.g. workflow.launchDir) so it isn't within the default results directory on the cloud but even that fails.
Steps to reproduce the problem
(Provide a test case that reproduces the problem either with a self-contained script or GitHub repository)
Program output
Environment
- Nextflow version: v26.04.6
- Java version: openjdk 23.0.2 2025-01-21
- Operating system: macOS
- Bash version: GNU bash, version 3.2.57
Additional context
(Add any other context about the problem here)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing .nextflow.log and the Google Batch executor path for an output directory backed by a GS bucket. Reproduce the failure with an enabled index scope, then verify that index files can be written to the GS output directory while local and AWS Batch behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, groovy
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100