OpenFn / OpenFn/lightning

Curated test inputs per workflow

Open
#5,132 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

As a project manager, I want to curate a set of test inputs for each workflow, so that developers can test against realistic data without ever touching production data.

Part of #4852 (Sandbox DevX epic).

Why this is the better answer

#5052 lets a developer carry the input from a failed run into a sandbox, scrubbed by hand. That is necessary, because sometimes only the real payload reproduces the bug. But it should be the exception, not the routine, because every use of it means a person deciding by eye what counts as sensitive.

If each workflow carries a small set of sanctioned test inputs, most work never needs production data at all. The developer opens a sandbox, picks from the suite, and gets on with it. Scrubbing a real run becomes the awkward case rather than the default one.

It also puts the decision in the right hands. Working out what is safe to hand a developer is a project manager's call, made once, rather than a developer's call made under time pressure on every bug.

Roughly what it looks like

A workflow has a named set of test inputs, curated by someone with the standing to decide what they contain. They can be written from scratch, or promoted from a scrubbed run dataclip once someone has done that work, so the effort is not repeated.

When a developer starts a sandbox for that workflow, the suite is what they see first. Choosing from it involves no review step, because the review already happened when the input was added to the suite.

Open questions

Where the suite lives. Per workflow reads well from the developer's side, but a project-level library with workflow tagging might match how people actually reuse inputs.

Who curates. Editor and above, or owner and admin only. This is really a question about who is accountable for what leaves production.

Whether a suite entry can be promoted from a scrubbed run, and if so whether that needs a second pair of eyes before it becomes reusable by everyone.

Whether these belong to the parent project and are inherited by sandboxes, or are copied in like other dataclips.

Not phase 1

Phase 1 needs the developer to be unblocked, which #5052 does. This is the thing that makes the safe path the default rather than merely available, and it deserves its own design pass rather than being rushed alongside.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reading the related issues #4852 and #5052, then resolve where curated inputs live, who can curate them, how scrubbed runs are promoted, and how sandbox inheritance works. Done means a settled design for making sanctioned workflow inputs the default without exposing production data.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend, developer-experience, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.