PRD: Replace comment suggestions with phased source-first ingestion

## Problem Statement

The SG Food Guide operator wants the directory to stay fresh without babysitting a complicated moderation workflow. Today, the newest Google Sheet entries and newest YouTube uploads are not reliably reflected quickly enough, the core stall sync is still sheet-first by design, and the existing comment suggestion/admin review flow adds substantial product and operational complexity without solving the main freshness problem. There is also no focused internal surface for answering basic operational questions such as whether the latest sheet row was seen, whether the latest channel upload was processed, why a candidate was skipped, or when a sync last succeeded.

## Solution

Replace the comment suggestion feature with a phased ingestion strategy centered on curated sources and operational visibility.

In phase 1, remove the comment suggestion product surface and runtime flow, preserve its database tables temporarily for rollback safety, add an admin-only ingestion status page, and move automated stall sync to an hourly cadence while keeping manual sync triggers.

In later phases, evolve stall ingestion so that Google Sheets and YouTube are both first-class inputs, add explicit per-row sheet freshness metadata, integrate a third-party transcript provider that works via server-side API key access, and allow YouTube-derived stalls to auto-publish only when transcript-backed parsing produces a high-confidence record with a name, country, and mappable place signal.

Matching should use normalized name plus mappable place signal plus country, with alias overrides for naming drift, and conflict resolution should be based on authoritative per-source timestamps so the newer source wins. The system should prefer false negatives over false positives.

## User Stories

1. As the site owner, I want the directory to reflect newly published source material within one hour, so that the product feels current.
2. As the site owner, I want to stop maintaining the comment suggestion workflow, so that engineering effort is focused on the ingestion path that matters.
3. As the site owner, I want the old comment feature removed from the product surface, so that the admin experience is simpler and less distracting.
4. As the site owner, I want existing comment-feature tables preserved for one release, so that rollback remains possible during the transition.
5. As an admin operator, I want an internal status page for ingestion health, so that I can tell whether the system is fresh without reading logs.
6. As an admin operator, I want to see the most recent successful and failed sync runs, so that I can quickly diagnose regressions.
7. As an admin operator, I want to see the newest timestamp observed from Google Sheets, so that I know whether sheet updates are being detected.
8. As an admin operator, I want to see the newest timestamp observed from YouTube uploads, so that I know whether channel freshness is being detected.
9. As an admin operator, I want to see unmatched videos and skipped candidates, so that I understand why fresh content did not publish.
10. As an admin operator, I want to see transcript fetch failures and low-confidence parse failures, so that I can distinguish source problems from matching problems.
11. As an admin operator, I want the status page behind existing admin access controls, so that operational details are not exposed publicly.
12. As an admin operator, I want to trigger sync manually in emergencies, so that I do not need to wait for the next scheduled run.
13. As an admin operator, I want scheduled sync to run hourly, so that the freshness goal is realistic.
14. As a maintainer, I want a shared trigger layer for scheduled and manual sync paths, so that behavior stays consistent across entrypoints.
15. As a maintainer, I want a dedicated ingestion status read model, so that status UI logic does not have to parse raw route state or duplicate database queries.
16. As a maintainer, I want Google Sheets and YouTube modeled as first-class ingestion sources, so that the system no longer assumes sheet rows are the only source of stall creation.
17. As a maintainer, I want an explicit `updated_at` field in sheet data, so that recency-based conflict resolution is deterministic.
18. As a maintainer, I want source conflicts to resolve using authoritative timestamps, so that the newer source can win without guesswork.
19. As a maintainer, I want transcript acquisition to happen through a server-side API-key-based provider, so that the solution works in the Cloudflare Workers deployment model.
20. As a maintainer, I want transcript availability to be a hard gate for YouTube-first publication, so that low-context video parsing does not silently corrupt the directory.
21. As a maintainer, I want publication rules to require a name, country, and mappable place signal, so that auto-published stalls meet a minimum quality bar.
22. As a maintainer, I want the system to prefer false negatives over false positives, so that it is safer to miss a candidate than to publish bad data.
23. As a maintainer, I want matching to use normalized name plus mappable place signal plus country, so that cross-source identity is more reliable than name plus cuisine alone.
24. As a maintainer, I want alias overrides for naming drift, so that the system can recover from imperfect automatic matching.
25. As a site visitor, I want newly added stalls to appear without obviously wrong merges or duplicates, so that I can trust the directory.
26. As a site visitor, I want YouTube-derived stalls to feel as structured as sheet-derived stalls, so that freshness improvements do not reduce quality.
27. As a site visitor, I want incorrect or low-confidence candidates to stay unpublished, so that the public directory remains curated.
28. As a site visitor, I want existing stall pages and search to continue working while ingestion evolves, so that backend changes do not degrade the core browsing experience.
29. As a maintainer, I want run history retained even after the comment feature is removed, so that transition-period debugging still has context.
30. As a maintainer, I want the PRD delivered as a phased roadmap, so that deletion, observability, and source unification can be shipped and validated incrementally.

## Implementation Decisions

- The PRD covers the full phased initiative, not only the first deletion milestone.
- The existing comment suggestion feature is treated as a separate bounded context and should be removed as code and product surface, not merely hidden from navigation.
- Comment-feature database tables and historical data should be preserved for one release before any destructive migration is considered.
- Phase 1 must include replacement observability; deletion is not considered complete without an admin-only ingestion status page.
- The ingestion status page should use a dedicated read model that aggregates recent sync runs, freshness indicators, warnings, failures, unmatched inputs, and skipped candidates.
- Admin status access should be implemented through thin server-side admin functions behind the existing Cloudflare Access admin boundary.
- Scheduled and manual sync entrypoints should share a common trigger/orchestration layer instead of duplicating request parsing, auth, and invocation behavior.
- Automated stall sync should move from daily cadence to hourly cadence in order to support the freshness target.
- Manual sync triggers should remain available as an operator override even after the status page ships.
- Google Sheets and YouTube should be treated as equal ingestion sources in the long-term architecture.
- A source-adapter approach should be used for sheets, YouTube metadata, and transcript retrieval so that source-specific logic is isolated behind stable interfaces.
- Google Sheet data must add an authoritative per-row `updated_at` field so that recency-based conflict resolution is reliable.
- When two sources disagree, the newer authoritative source should win based on persisted source timestamps.
- Transcript retrieval should rely on a third-party provider that exposes a clean server-side API using an API key, with no browser cookies or personal account coupling.
- Transcript availability is a hard requirement for YouTube-first publication; if transcript fetch fails, the candidate should remain unpublished.
- YouTube-derived stalls may publish only when parsing yields a high-confidence record with at least a stall name, country, and mappable place signal.
- The system should bias toward false negatives rather than false positives.
- Cross-source canonical matching should use normalized name plus mappable place signal plus country.
- Alias overrides should be part of the canonical matching design as a safety valve for naming drift and imperfect automatic matches.
- The status surface should remain internal/admin-only rather than becoming a public diagnostics page.
- The solution should preserve existing public browsing functionality while the ingestion backend is refactored.
- Deep modules the implementation should revolve around include: an ingestion status read model, thin admin status functions, a shared sync trigger layer, source adapters, a canonical matching module, and alias override support.

## Testing Decisions

- Good tests should verify external behavior and stable contracts, not route-local implementation details or incidental state structure.
- Tests should use deterministic fixtures and mocked network boundaries so that matching, publication, and freshness behavior are reproducible.
- The PRD should explicitly require tests for matching and publication rules, including transcript-gated publication, confidence thresholds, conflict resolution, and alias override behavior.
- The PRD should explicitly require tests for the ingestion status read model and admin access behavior.
- The PRD should explicitly require tests for scheduler and manual trigger behavior, with coverage focused on observable outcomes rather than framework internals.
- Existing prior art for admin access tests is the current Cloudflare Access authorization test suite.
- Existing prior art for runtime/context boundary tests is the current Cloudflare runtime test suite.
- Existing prior art for API handler tests is the current server API test style that stubs fetch and asserts on HTTP status and JSON responses.
- Existing prior art for thin loader tests is the current route-loader test style.
- The implementation should prefer testing deep modules directly rather than trying to exhaustively test large route UI files.

## Out of Scope

- Immediate destructive deletion of legacy comment-feature database tables.
- A public-facing ingestion diagnostics page.
- Browser-based transcript scraping, cookie-based extraction, or shell-based tools that do not fit the Worker deployment model.
- Guaranteed publication of every new YouTube upload regardless of transcript availability or parse confidence.
- A redesign of unrelated public product features beyond removing the old comment/admin surfaces and adding the replacement internal status page.
- A full redesign of unrelated review-sync functionality.

## Further Notes

- The roadmap should be explicitly phased. Phase 1 is deletion of the comment suggestion product/runtime surface plus the admin-only ingestion status page, while preserving historical data and keeping manual triggers. Later phases introduce source unification, sheet freshness metadata, transcript-backed YouTube ingestion, stronger matching, and timestamp-based conflict resolution.
- The current codebase already persists sync run summaries; the main missing capability is a read-side model built for operators.
- Because the project prefers false negatives, low-confidence or under-specified YouTube candidates should remain out of the public directory until they meet the publication threshold.