apache / apache/texera

Separate state and statistics as distinct sub-concepts in WorkflowStatusService

Closed
#5,919 1 comment 0 reactions 1 assignee Claimed by @PG1204 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

Formalize `WorkflowStatusService`'s two existing concerns, operator **state** and operator **statistics**, as cleanly separated sub-concepts, rather than bundling them in a single `OperatorStatistics` object.

### Context

`WorkflowStatusService` currently carries two distinct concepts in one shape: `OperatorStatistics` mixes `operatorState` together with the per-operator statistics (row counts, sizes, timing). The RFC discussion (#5216) noted these are really separate sub-concepts and should be formalized as such before/ alongside layering performance metrics on top.

This was raised during review of #5834 (which introduced performance metrics as a cleanly separated concept). To keep that PR scoped to capturing metrics, and to avoid adding API with no consumer yet and the state/statistics separation was deferred and agreed to be handled as a dedicated PR.

Before: OperatorStatistics { operatorState, ...statistics } // two concepts, one blob
After: state -> Record
statistics -> Record // statistics only
performance -> Record // already separate

### Proposed Change

In `frontend/src/app/workspace/service/workflow-status/`:

- Split the `state` concept out of `OperatorStatistics` so the statistics type holds metrics only, and expose operator state as its own concept on `WorkflowStatusService` (e.g. a dedicated state stream + snapshot).
- Keep the existing public API working (or migrate consumers) so the change is behavior-preserving; update call sites that currently read `operatorState` off the combined object.
- Performance metrics (`OperatorPerformanceMetrics`) remain the separate third concept introduced in #.

The exact API shape (separate streams vs. a structured container) to be settled in the PR; the goal is three clearly separated concepts: state, statistics, performance.

### Required Test

- Characterization tests first to pin current behavior (this is a refactor with no behavior change), then update the affected specs.
- `WorkflowStatusService` spec: assert state and statistics are exposed and update independently; existing consumer specs stay green.

### Related

- Umbrella issue #5772
- Follow-up from #5834 (review discussion)
- RFC discussion #5216

### Task Type

- [x] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.