apache / apache/texera

Workflow Snippets: reusable operator bundles for common pipeline patterns

Open
#5,165 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

During the hackathon I prototyped Workflow Snippets: reusable operator groups that users can drag onto the canvas as a bundle. (Reference: PR #5117)

Users frequently rebuild the same operator chains from scratch — CSV File Scan → Filter → Remove Duplicates for data cleaning, Train/Test Split → Logistic Regression → **Sklearn** Testing for classification. There's no way to save a group of connected operators as a reusable template.

Workflow Snippets lets users save and reuse common operator bundles. Instead of dragging operators one by one and connecting them manually, users drag one snippet onto the canvas and get a fully connected operator chain in a single action.

This is a UI-level feature: each operator in the snippet stays independent on the canvas (unlike Macro Operators #5115, which merge operators into a single engine-level node). Snippets and Macros solve different problems and coexist naturally. Reference prototype: PR #5117

### Proposed Solution or Design

### What is a snippet?

A snippet is a small piece of workflow JSON, the same format as `workflow.content`, just containing **a few operators and their links** instead of an entire workflow.

---

### How to create a snippet?

**Path 1 : From the canvas:**
> Select 2+ connected operators → right-click → **"Save as Snippet"** → enter a name → saved.
> Reuses the existing `OperatorMenuService.copyOperator()` serialization format.

**Path 2 : From a dedicated Snippets page:**
> Sidebar → Your Work → **Snippets** → "+ Create Snippet" → pick operators from the catalog → arrange in order → save.
> This lets users build snippets **without having a workflow open**.

---

### How to use a snippet?

The operator panel gets a new **"Snippets"** section at the bottom, listing saved snippets.

Drag a snippet onto the canvas:
1. System creates all operators with **fresh IDs** (reusing `assignNewOperatorIds()`)
2. Connects them automatically
3. Wrapped in one **undo/redo** action via `WorkflowActionService.addOperatorsAndLinks()`

---

### How to store?

New `workflow_snippet` + `snippet_user_access` tables, following the existing `workflow` / `workflow_user_access` pattern. Backend CRUD via a new `SnippetResource` mirroring `WorkflowResource`.

### How to share?

Toggle a snippet as **"public"** → other users can browse and use it.

---

### Suggested PR breakdown

1. Database tables + backend CRUD (`SnippetResource`)
2. Snippets page: create and manage snippets
3. Snippets section in operator panel + drag-to-canvas
4. Right-click "Save as Snippet" from canvas

---

demo purpose:

Image

**Would appreciate feedback on this one.**

### Affected Area

Workflow UI

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the prototype in PR #5117 and the named entry points: OperatorMenuService.copyOperator(), assignNewOperatorIds(), WorkflowActionService.addOperatorsAndLinks(), and WorkflowResource. Compare the proposed workflow_snippet and snippet_user_access tables with the existing workflow access pattern. Done means the four suggested creation, management, insertion, and sharing paths work as one coherent feature.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.