Process idea: field testing a QEP while in Draft
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 1
- Forks
- 0
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 7
Description
Idea
Add field testing to the QEP process: once a QEP is in Draft, run one or more real decisions against the draft text and record the outcome in the discussion PR, before the decision deadline. This would land as an in-place amendment to QEP-1's "How a QEP is decided" section if adopted.
Where the idea comes from
QEP-3 (#7) got two rounds of field testing during its comment window, and both were more informative than any amount of reading the text:
- A validation. A live placement decision (
project-reportingvsworkspace-reporting) was run against the draft's goal-vs-fleet boundary rules, which decided it without interpretation. That is direct evidence the rules work in practice — exactly what a reviewer wants to know before accepting a standard, and something a read-through cannot establish. - A defect found while the draft was still cheap to change. Pulling on one thread the test exposed (
reports-activitydoesn't parse under the registry) revealed that theworkflow-*type names a mechanism rather than a role, and produced a concrete restructure proposal (reporter-*/task-*) before acceptance rather than as a post-acceptance amendment. - An earlier field report on the same PR (the translation program's bench placement) surfaced two text gaps that were fixed in-flight (8150620).
The pattern generalises: a standard-type QEP exists to settle recurring decisions, so the natural test of a draft is to settle one real decision with it and see whether the text was sufficient. Every outcome is useful — a clean decision becomes validation evidence, a strained one becomes an amendment while amending is free.
Proposal sketch
Amend QEP-1's decision process with an optional step between Draft and Decide:
(Optional, encouraged for standards) Field test. Apply the draft to at least one real decision it claims to govern, and record the outcome as a field report comment on the discussion PR: what was decided, whether the text sufficed, and any amendments it motivated. Field reports are review evidence at the deadline.
Deliberately lightweight: no template, no gate, no new status — just a named, expected activity so authors and reviewers know it counts as review.
Points to decide
- Optional vs expected. Fully optional, or expected for
type: standardQEPs (which govern recurring decisions and therefore always have a testable case) while remaining optional for process/informational QEPs? - Interaction with the deadline. Leave the one-to-two-week window unchanged, or note that a substantive amendment arising from a field test should reset a short fresh window (as is happening on #7)?
- Who runs it. The author testing their own draft is still valuable (both #7 rounds were author-run), but a field test by a second maintainer is stronger evidence; worth encouraging without requiring.
- Naming. "Field test" / "field report" as used on #7, unless someone prefers another term.
If there's support, I'll draft the QEP-1 amendment (version bump per QEP-1's own rules) as a PR.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read QEP-1's "How a QEP is decided" section and compare the field reports from QEP-3 (#7), including commit 8150620. Resolve the four proposal points, then draft the QEP-1 amendment and required version bump if the project supports it. Done means the process change and its rationale are documented for review.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100