apache / apache/texera

Add the sample-value config framework and validation

Open
#7,525 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

Step 7 of the operator-demo video generator: add the layer that holds each operator's sample values and validates them in CI.

When an operator is introduced or changed, its author maintains the sample values used to generate that operator's video, and generates the video once to confirm it looks right. So the values need a defined place to live, plus a check that catches a stale entry at build time rather than at record time.

### Proposed Solution or Design

Add the configuration and validation needed for operator demo videos.

- `TestDataConfig`
Stores the shared test setup used when generating videos, such as the Texera URL, demo account, sample datasets and files, workflow templates, and where generated videos should be saved.

- `OperatorFieldValues`
Loads the sample values used to fill each operator's form. It also keeps demo-specific settings, such as which dataset or file an operator should use, separate from the operator's actual form fields.

- `OperatorFieldValuesValidator`
Checks that the sample values are still valid for the current operator definition. For example, it catches:

* fields that were renamed or removed,
* values in the wrong format,
* required fields that are missing,
* dataset columns that do not exist,
* columns with incompatible data types.

CI: Run this validation in CI. If an operator change makes its demo configuration invalid, the PR should fail with a clear message showing which operator and field need to be updated.

### Affected Area

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing operator definitions, demo-video generation flow, and CI configuration, then determine where TestDataConfig, OperatorFieldValues, and OperatorFieldValuesValidator belong. Done means sample values can be maintained per operator, invalid fields and dataset mappings produce clear validation errors, and the validation runs in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
ci-cd, testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.