nextflow-io / nextflow-io/nextflow
Google Batch: share a single getTaskStatus lookup in getTraceRecord()
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Background
Follow-up recorded during the review of #7629 (non-blocking there, tracked here).
GoogleBatchTaskHandler.getTraceRecord() makes two separate client.getTaskStatus(jobId, taskId) calls per task, both fetching the same object:
getMachineInfo()→updateZoneFromEvents()→client.getTaskStatus(...)to resolve the actual execution zone from the status eventsgetNumSpotInterruptions(jobId)→client.getTaskStatus(...)to count preemption events (exit code 50001)
Both read statusEventsList off the same response.
Why it matters now
Before #7629, array-child handlers had a null machineInfo, so getMachineInfo() returned early and never reached updateZoneFromEvents() — only the spot-interruption lookup ran. Now that children carry a non-null machineInfo, they perform the zone resolution too. That is the desired behaviour (each child resolves its own actual zone rather than inheriting the parent's planned one), but it doubles the number of Batch API calls at trace time. On a large array job that is one extra getTaskStatus() per task.
Proposal
Fetch the task status once in getTraceRecord() and share it between the zone resolution and the spot-interruption count, e.g. pass the already-fetched status (or its statusEventsList) into both paths instead of having each one call the client. resolveZoneFromEvents(List<StatusEvent>) is already factored to take events directly, so the zone side is a small change; getNumSpotInterruptions() would need an overload accepting the status/events.
Both call sites already tolerate a null status, so the shared lookup should keep the same defensive handling.
References
plugins/nf-google/src/main/nextflow/cloud/google/batch/GoogleBatchTaskHandler.groovy—getTraceRecord()(~L910),updateZoneFromEvents()(~L838),getNumSpotInterruptions()(~L878)- Review: https://github.com/nextflow-io/nextflow/pull/7629#pullrequestreview-5236250185
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in plugins/nf-google/src/main/nextflow/cloud/google/batch/GoogleBatchTaskHandler.groovy by reading getTraceRecord(), updateZoneFromEvents(), and getNumSpotInterruptions(). Trace how statusEventsList is consumed and check the existing Google Batch handler tests, if available. Done means each task performs one status lookup while preserving zone resolution, interruption counting, and null handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, groovy
- Domain
- cloud
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100