apache / apache/texera

refactor(agent-service): redesign the sync-execution result and error model

Open
#5,750 0 comments 0 reactions 1 assignee Claimed by @bobbai00 View on GitHub
agent-service refactor
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

Redesign the sync-execution result/error model. Replace the flat `OperatorInfo` with `OperatorExecutionSummary` (separating state, errors, result, and console-message projections), make `sampleTuples` a list of `SampleRow { rowIndex, tuple }` instead of embedding `__row_index__` in the tuple, and reuse the engine's proto `WorkflowFatalError` for per-operator errors so compile and execution errors share one wire shape.

Current problems in the existing code:

- `OperatorInfo` mixes unrelated concerns in one flat interface: execution state, tuple counts, input shapes, result payload, truncation/display flags, console logs, warnings, errors, and statistics.
- Many fields are optional (`result?`, `inputPortShapes?`, `totalRowCount?`, `displayedRows?`, `truncated?`, `consoleLogs?`, `error?`, `warnings?`, `resultStatistics?`), so consumers must infer semantics from missing fields instead of explicit fields with clear meanings.
- Result rows carry internal metadata such as `__row_index__`, which pollutes user tuple payloads and forces each consumer to filter special keys.
- Error shapes are inconsistent: compilation errors use `WorkflowFatalError`, per-operator execution errors use bespoke strings, and workflow-level errors use a separate string list.
- Backend, agent-service, REST, and frontend types overlap but are not aligned, which makes the wire contract harder to reason about and easier to drift.

Part of #5747. Implemented by #6009. Supersedes #5927.

### Task Type

- [x] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.