AletheiaFact / AletheiaFact/aletheia

[Epic] Process & operations — real bottlenecks in the fact-checking cycle

未关闭
#2,490 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
backlog epic needs-investigation
主要语言
TypeScript
星标
55
派生
20
平均合并
2 天 6 小时
30 天内合并 PR
2

描述

**Epic B · Process & operations** — the real bottlenecks in the fact-checking cycle.
_Derived from the fact-checking cycle analysis (v2, 2026-07-03). Production numbers are namespace `main` unless noted._

# 🎯 Strategic Context

## Why Now?
> _Owner: CTO_

The analysis relocated the bottleneck. The time in a fact-check is **not** in the work — research and assignment are near-instant. It is in **approval → publication** (the single largest median stage), in a **queue stalled for ~8 months**, and in a **fact-checker base so concentrated that one account holds 222 initial reviews**. These are operational problems we can act on now, and two of them need no new engineering to *start*. As we scale to 10 hubs, an unmanaged queue and a single-point-of-failure reviewer base become existential, not cosmetic.

## Problem Statement
> _Owner: CTO, Design Lead, PE_

#### For Fact-Checking Teams (front-line reviewers):
Reviewers carry a backlog they cannot see the shape of: ~215 open tasks, many idle for months, with no signal about which are salvageable. New reviewers churn — **39% do exactly one activity and never return** — so the load concentrates on a handful of people, which is stressful and fragile. A finished check then waits in approval far longer than it took to research.

#### For Platform Operators & Institutional Partners (hubs):
Operators have no triage policy, WIP limit, or SLA, so the queue only grows. The largest delay — approval → publication (**~66 h median in `main`, ~48 h overall**, above cross-checking's ~27 h) plus a **~12-day gap** between internal `published` and the public claim review — sits in a process step nobody owns explicitly. Cross-checking has genuinely slowed (**~6.6 h in 2024 → ~69.8 h in 2025–26, `main`-only**), which may signal a capacity problem a partner would feel directly.

#### Impact:
- The stalled queue **dominates ~78% of wall-clock time** in the pipeline.
- ~60–75 tasks are effectively abandoned (empty `reviewData`, idle ~590–660 d); ~142–155 are genuinely stuck-active (idle ~226–235 d; `main`-only ~108→70 d).
- Retention/concentration risk: **48 of 122** all-time fact-checkers are one-off; the operation leans on ~15 active/month with one dominant account.

## Solution Hypothesis
> _Owner: CTO, Design Lead, PE_

Attack the three bottlenecks directly: (1) **map and shorten approval→publication** — since the time isn't in execution, much of it is likely editorial hand-off, scheduling, or manual QA that can be reduced or automated; (2) **govern the backlog** by decomposing it (abandoned → archived by design; stuck-active → triage + SLA/WIP); (3) **fix the first-task funnel** to convert one-off reviewers into returning ones. **Evidence:** the near-zero research/assignment times prove the delay is process, not effort; the clean abandoned-vs-stuck split proves most of the "217 backlog" is dead weight removable by design. **Assumption:** the approval delay is process/queue, not reviewer availability (to be confirmed by PROC-1's mapping).

## What We're Claiming (Narrative)
> _Owner: CTO_

A fact-check that's ready to publish now goes live in a fraction of the time it used to wait, because we found and fixed the hand-off that was quietly holding it. Our queue is a managed backlog, not a graveyard — stale work is triaged, reassigned, or retired instead of sitting invisibly for months. And a new volunteer's first check is a guided, finishable experience, so more of the people who show up once come back. The platform now runs like an operation, not a pile-up.

## Success Outcomes
> _Owner: CTO, Product Ops, PE_

Targets are benchmarked to current baselines. Some are **investigation-gated** — PROC-1 and PROC-4 must first find the cause before a reduction target is committed; those are marked accordingly and measured once Epic A instrumentation lands.

| Outcome | Type | Target | Why Realistic | Measurement | Comments |
| --- | --- | --- | --- | --- | --- |
| Approval→publication latency | Quant | Cut median ≥ 30% from ~48–66 h baseline (final target set after PROC-1 mapping) | Time is process, not execution → largely automatable | `stageDurations` (via INST-3/INST-2) | Investigation-gated |
| Abandoned tasks removed from WIP | Quant | 100% of ~60–75 abandoned archived; 0 in live WIP | Deterministic rule (empty `reviewData` + idle) | DB query on `lifecycle.status` (INST-4) | Archive by design |
| Stuck-active backlog bounded | Quant | ≥ 80% of ~142–155 triaged (resumed/reassigned/rejected) within one SLA window | Finite, known set | DB query pre/post triage | Needs INST-4; mechanism TECH-4 |
| One-off reviewer rate | Quant | Reduce from 39% toward < 30% | Small funnel changes move first-task completion | INST-7 onboarding funnel | Measured once INST-7 ships |
| Cross-check slowdown diagnosed | Qual | Root cause identified before any target is set | Real slowdown confirmed `main`-only | PROC-4 investigation + INST-2 | Avoid optimizing the wrong stage |
| Concentration risk | Quant | > 1 account handling a meaningful share of initial reviews | Broadening the base is the explicit goal | Reviews-per-user distribution | Baseline: 1 account = 222 initial reviews |

---

# 📋 Scope and Implementation Plan
> _Owner: CTO, Product Lead, Design Lead, Engineering Lead_

### Epic Release Timeline
PROC-1 and PROC-2b can **start now**; the rest need Epic A instrumentation only to *measure* the effect, not to begin.

- [ ] #2499 — PROC-1 · Shorten the Approval → Publication stage — 🔴 — **start now** (measure via INST-3 #2493)
- [ ] #2500 — PROC-2 · Stalled backlog — **umbrella**, split into:
- [ ] #2509 — PROC-2a · Purge/diagnose the ~60–75 abandoned tasks — 🔴 (needs INST-4 #2494)
- [ ] #2510 — PROC-2b · Triage/SLA the ~142–155 stuck-active tasks — 🔴 (mechanism TECH-4 #2514)
- [ ] #2501 — PROC-3 · Fact-checker retention/onboarding — 🟡 (measure via INST-7 #2497)
- [ ] #2502 — PROC-4 · Investigate cross-checking slowdown — 🟡 (precise read via INST-2 #2492)

**Sequencing & dependencies:** PROC-2a/2b depend on INST-4 for the `abandoned`/`archived` state; PROC-2b's mechanism is TECH-4 (SLA + auto-owner). PROC-1's effect is measured via INST-3; PROC-3's via INST-7; PROC-4's precise read via INST-2. Reconcile PROC-4 wording with INST-11 (see the D1 note in #2502).

> 💡 For ideas outside this scope, open a feature request so we can track and revisit them in a future iteration.

---

# ❓ Outstanding Questions
> _Owner: CTO, PE, Lead Engineer, Design Lead, Product Ops, Data Lead_

**Q:** What actually happens between `submitted` and the public claim review — editorial approval, scheduling, or manual QA? (PROC-1 must map this before we set a latency target.)
**Q:** Is the cross-check slowdown driven by fewer available cross-checkers, more complex tasks, or a process change? (PROC-4.)
**Q:** What idle threshold and SLA window define "abandoned" vs "triage now" for the daily job? (Depends on INST-4.)
**Q:** Should archived/abandoned tasks be purged, or retained read-only for audit/IFCN?

---

# ⚠️ Risks
> _Owner: PE, Lead Engineer_

| Risk | Likelihood | Impact | Mitigation |
| --- | --- | --- | --- |
| Auto-expiry/reassignment silently drops legitimately-paused work | Medium | High | Notify + grace period before reassignment (TECH-5 orphan alerts); reversible archive, never hard delete |
| Approval delay turns out to be reviewer availability, not process | Medium | Medium | PROC-1 mapping first; keep the target investigation-gated |
| Shortening approval bypasses an editorial/QA safeguard | Low | High | Map the QA intent before automating; graduate via namespace trust (TECH-6) |
| Retention fixes can't be measured until INST-7 ships, delaying feedback | Medium | Low | Start funnel changes now; treat pre-INST-7 signal as qualitative |

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。