Investigate: structured feedback classification as a cross-cutting protocol

Aperta
#167 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

@abeltrano ci sta già lavorando.

Dal 2/4/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

enhancement

Motivation

PR #166 added a feedback classification rule to the spec-extraction-workflow template (Phase 3) that requires the LLM to categorize user feedback as Editorial, Correction, or Finding — defaulting to Finding when uncertain. This prevented a real failure mode where domain-expert feedback was silently under-promoted to a cosmetic change.

The underlying problem — LLMs default to the lowest-impact interpretation of user input — is not specific to spec extraction. Any interactive template where the user provides feedback mid-workflow is susceptible.

Investigation Questions

  1. Which templates have interactive feedback loops? Candidates include:

    • interactive-design (Phase 1 reasoning loop)
    • extend-library (contributor guidance workflow)
    • engineering-workflow / collaborate-requirements-change (iterative change propagation)
    • evolve-protocol (protocol modification loop)
    • author-north-star (section-by-section drafting with user review)
    • author-presentation (multi-phase with user review)
    • maintenance-workflow (finding classification with user collaboration)
    • Any template using the iterative-refinement protocol
  2. Should this be a protocol rather than per-template guidance? If the pattern applies broadly, it may belong as:

    • A new guardrail protocol (e.g., feedback-classification) applied to all interactive templates
    • An addition to the existing iterative-refinement protocol (since that protocol already governs how changes are applied during feedback loops)
    • A section in the anti-hallucination protocol (since under-promoting feedback is a form of information loss)
  3. Does the classification taxonomy need to be richer for other contexts? The current three categories (Editorial / Correction / Finding) were designed for spec extraction. Other templates might need:

    • Scope change — user wants to expand or narrow the task
    • Constraint — user is adding a new constraint not previously stated
    • Preference — user is expressing a stylistic or approach preference (not a requirement)
    • Rejection — user is rejecting a proposed element entirely
  4. What is the interaction with existing protocols?

    • iterative-refinement already says 'justify every change' — should feedback classification be embedded there?
    • requirements-elicitation already decomposes user input — should it also classify the input type?
    • adversarial-falsification already has confidence classification — is there overlap?
  5. What form should it take?

    • A standalone guardrail protocol (protocols/guardrails/feedback-classification.md)
    • An extension to iterative-refinement protocol
    • Guidance in bootstrap.md for all interactive templates
    • A combination (protocol + bootstrap guidance)

Suggested Approach

  1. Audit all interactive templates and the iterative-refinement protocol for feedback handling gaps
  2. Draft a candidate protocol or protocol extension
  3. Test it against 2-3 real interactive sessions to validate the taxonomy
  4. Propose via extend-library template contribution workflow

Related

  • #165 — Original issue (spec-extraction-workflow specific fix)
  • #166 — PR implementing the spec-extraction-specific classification rule
Lingua principale
JavaScript
Stelle
105
Fork
23
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/PromptKit

Tutte le issue di microsoft/PromptKit

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.