Curated test inputs per workflow
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
As a project manager, I want to curate a set of test inputs for each workflow, so that developers can test against realistic data without ever touching production data.
Part of #4852 (Sandbox DevX epic).
Why this is the better answer
#5052 lets a developer carry the input from a failed run into a sandbox, scrubbed by hand. That is necessary, because sometimes only the real payload reproduces the bug. But it should be the exception, not the routine, because every use of it means a person deciding by eye what counts as sensitive.
If each workflow carries a small set of sanctioned test inputs, most work never needs production data at all. The developer opens a sandbox, picks from the suite, and gets on with it. Scrubbing a real run becomes the awkward case rather than the default one.
It also puts the decision in the right hands. Working out what is safe to hand a developer is a project manager's call, made once, rather than a developer's call made under time pressure on every bug.
Roughly what it looks like
A workflow has a named set of test inputs, curated by someone with the standing to decide what they contain. They can be written from scratch, or promoted from a scrubbed run dataclip once someone has done that work, so the effort is not repeated.
When a developer starts a sandbox for that workflow, the suite is what they see first. Choosing from it involves no review step, because the review already happened when the input was added to the suite.
Open questions
Where the suite lives. Per workflow reads well from the developer's side, but a project-level library with workflow tagging might match how people actually reuse inputs.
Who curates. Editor and above, or owner and admin only. This is really a question about who is accountable for what leaves production.
Whether a suite entry can be promoted from a scrubbed run, and if so whether that needs a second pair of eyes before it becomes reusable by everyone.
Whether these belong to the parent project and are inherited by sandboxes, or are copied in like other dataclips.
Not phase 1
Phase 1 needs the developer to be unblocked, which #5052 does. This is the thing that makes the safe path the default rather than merely available, and it deserves its own design pass rather than being rushed alongside.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reading the related issues #4852 and #5052, then resolve where curated inputs live, who can curate them, how scrubbed runs are promoted, and how sandbox inheritance works. Done means a settled design for making sanctioned workflow inputs the default without exposing production data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, developer-experience, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100