[Ticket]: 7 — Final Validation & Recommendation (Days 13–14)

### Type
Documentation

### Summary
We need to validate the selected mitigation (Ticket 6) on **unseen test data** and produce the final recommendation, because a solution proven only on the evaluation set it was tuned on is not proven at all. Expected result: a final evaluation report with final metrics, the recommended configuration, remaining limitations, a reproduction guide, and production deployment recommendations.

### Intent
We must answer "what should we change in the deployed system and what improvement can we demonstrate?" — the definitive, evidence-backed conclusion of the epic that the business can act on.

### Source of truth (links)
- Epic #1062 — Deep Analysis & Mitigation of Qwen3.5-2B Hallucinations (this ticket is its Ticket 7).
- Requester-provided epic specification (2026-08-31), Ticket 7 — "Final Validation & Recommendation".
- Ticket 6 — best-performing configuration and trade-off analysis.
- Ticket 3 — original baseline metrics.
- `impl.md` — frozen deployment configuration (the "before" reference).
- `docs/hallucinations/` — protocol, taxonomy, annotation rules.

### Current Behavior
- A candidate configuration exists but has only been evaluated on the development evaluation set; its generalization is unknown.

### Expected Behavior
- **Final configuration run on an independent (unseen) evaluation set.**
- **Comparison against the original baseline** (Ticket 3 metrics) on the unseen set.
- **Verified hallucination reduction** and **verified accuracy not significantly degraded**.
- **Latency/throughput impact measured**.
- **Remaining hallucinations analyzed** (residual error analysis by taxonomy/severity).
- **Limitations documented**.
- **Final technical report** containing:
- Final evaluation report and final metrics.
- Recommended configuration (concrete: sampling parameters, prompt/flow changes, any grounding/verification/abstention step).
- Remaining limitations.
- Reproduction guide (how to re-run the benchmark, runner, and analysis).
- Recommendations for production deployment (rollout plan, monitoring hooks, success gates).

### Acceptance Criteria
- [ ] Given the independent evaluation set, when the final configuration runs, then the dataset was not used during Tickets 3–6 (truly unseen).
- [ ] Hallucination reduction is demonstrated vs the original baseline on the unseen set.
- [ ] Accuracy is verified not significantly degraded (statistical check reported).
- [ ] Latency/throughput impact is measured and reported.
- [ ] Remaining hallucinations are analyzed by category and severity.
- [ ] Limitations are documented honestly (including AI-usage caveats).
- [ ] The recommended configuration is concrete and actionable.
- [ ] A reproduction guide is provided (end-to-end).
- [ ] Production deployment recommendations include rollout and monitoring gates.
- [ ] Verification evidence is provided (report + metrics + reproduction steps).

### Out of Scope
- Executing the production rollout (recommendation only; rollout is a follow-up ticket).
- Further mitigation design.
- Model changes.

### Technical Context
- Independent evaluation set: must be disjoint from the Ticket 2 benchmark subset used in Tickets 3–6; version and store it with the report.
- Final metrics reuse the Ticket 3 definitions; hallucination reduction uses the Ticket 6 formula.
- Report location: `docs/hallucinations/` (e.g. `report-qwen3-5-2b.md`), following repo documentation conventions.
- Production recommendation must respect the deployment constraints (single GPU, `minBackends: 1`, cluster-local federation, gateway rate limits).

### Risks
- Unseen set not truly independent (leakage). Mitigation: construct/version the unseen set before Ticket 7; verify no overlap.
- Improvement is benchmark-specific. Mitigation: report residual errors and generalization caveats.
- Recommended config conflicts with serving constraints. Mitigation: validate feasibility against `impl.md` and chart values.

### Test Plan
- Run the final configuration on the unseen set (fixed seed, N runs).
- Recompute baseline metrics on the same unseen set for a fair comparison.
- Statistical check of accuracy difference (e.g. significance or confidence interval).
- Measure latency/throughput (gateway metrics + per-inference).
- Walk through the reproduction guide from a clean state.

### Verification evidence
- Final technical report (metrics, recommended configuration, limitations, reproduction guide, production recommendations).
- Unseen-set results table vs baseline.
- Latency/throughput impact numbers.
- Residual hallucination analysis.

### Human accountable owner
@stephane-segning

### AI Usage Declaration
Drafting the ticket, Writing documentation

### Human verification completed
- [ ] I understood the intent
- [ ] I checked the source of truth
- [ ] I reviewed all AI-generated text/code
- [ ] I verified the implementation manually
- [ ] I verified the tests
- [ ] I checked for hallucinated assumptions
- [ ] I documented remaining risks
- [ ] I am the accountable owner and accept responsibility for this ticket.