koala73 / koala73/worldmonitor
chore(news): story-identity hardening follow-ups from the #4924 adversarial review — adopt-existing-track canonical, feature-hash salt, corroboration exposure on brief tools
- Dominant language
- TypeScript
- Stars
- 86.6k
- Forks
- 13.1k
- Avg merge
- 8h 4m
- Merged PRs (30d)
- 825
Description
Residuals from the PR #4924 multi-agent + cross-model review (all applied findings landed in that PR; these are the deliberately-deferred items).
1. **Adopt-existing-track canonical (adversarial P1 residual).** The canonical cluster id anchors on the earliest-published member (fixed in #4924), but RSS `publishedAt` is publisher-controlled: a hostile feed can backdate within the 96h freshness window to claim a live story's identity, then rotate titles to churn its story:track row (phase re-fires, counts reset). Hardening: before minting a canonical, batch-check EXISTS on member exact-hash story:track rows and adopt the oldest EXISTING track's hash — server-side Redis state a feed cannot fake. Cost: one EXISTS pipeline (~batch size) per digest build; wire into assignStoryIdentity via an injected `resolveExistingTracks` dep to keep the module pure.
2. **Feature-hash salt (adversarial residual, theoretical).** FNV-1a at DIM=512 is unkeyed and public — offline collision search could craft gibberish tokens that project onto a target story's dimensions. A per-deploy salt (env-derived seed folded into both fnv1a seeds) breaks offline search; costs nothing at runtime. Must invalidate no persisted state (vectors are never stored).
3. **Expose the now-trustworthy corroboration to agents (agent-native).** `get_world_brief`/`get_country_brief` (api/mcp/registry/rpc-tools.ts:159-169, :254-267) strip `corroborationCount` and `storyMeta.phase` from the items they fetch — agents can't distinguish a 6-source corroborated story from a single unconfirmed claim, while the NewsPanel UI shows exactly that. Add both fields to the sources/topStories arrays those tools return. Also: `get_news_intelligence`'s outputSchema under-documents the enriched fields that already pass through (`sourceCount`, `corroborationSourceCount`, `memberTitles`).
4. **Candidate-token stopword list (performance advisory).** candidateTokens has no stopword filter; ultra-hot buckets are capped at 250 (#4924), but a small connector-word denylist would restore index selectivity if batch sizes grow past the current ~1500 envelope.
Contributor guide
Research direction
Start at assignStoryIdentity and the injected resolveExistingTracks dependency, then inspect the feature-hash implementation and api/mcp/registry/rpc-tools.ts:159-169 and :254-267. Confirm the deferred identity, salting, corroboration, and stopword requirements, including the get_news_intelligence outputSchema fields. Done means existing tracks are adopted, feature hashes are salted without persisted-state invalidation, brief tools expose corroboration and phase, and the enriched schema is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- redis, typescript
- Domain
- api, backend-api-design, performance, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100