apache / apache/texera

Move cluster-induced workflow shutdown from engine into web

Open
#5,432 0 comments 0 reactions 1 assignee Claimed by @Yicong-Huang View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

`amber/src/main/scala/org/apache/texera/amber/clustering/ClusterListener.scala` listens to Pekko cluster events and, when a node leaves, iterates `WorkflowService.getAllWorkflowServices` to call its own `forcefullyStop(executionService, cause)` on each in-flight execution. Both calls reach into web (`web.service.WorkflowService`, `web.service.WorkflowExecutionService`, `web.storage.ExecutionStateStore`).

After the engine event work lands, `ClusterListener` only needs to publish an engine event (e.g. `ClusterNodeRemoved`); the web-side subscriber handles enumeration of active workflows and the force-stop business logic. Move the body of `forcefullyStop` and the `WorkflowService` iteration to that web-side subscriber. `ClusterListener` ends up as a thin "Pekko cluster → engine event" bridge with no web imports.

Part of #5424. Depends on the engine-event proto sub-issue.

### Task Type

- [x] Refactor / Cleanup

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.