feat(agent): live Argentine news crawler & spatio-temporal graffiti agent
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
## Description
Once Handshake is live, deploy an autonomous live agent / daemon that continuously crawls and scrapes Argentine news outlets (national & regional), extracts spatial and temporal event context, and signs & publishes them as real-time **geo-temporal graffitis** on the Handshake map.
Users opening Handshake will see live breaking news, local events, traffic/protests, and cultural updates physically anchored to the exact locations in Argentina where the events are happening.
---
## Architecture & Pipeline
### 1. Ingestion & Web Scraping Engine
- Monitor RSS feeds, sitemaps, and live pages of national outlets (*Infobae, Clarín, La Nación, Página/12, Ámbito, Télam/Somos Télam, etc.*) as well as regional Argentine media (*La Voz, El Día, La Capital, etc.*).
- Filter recent articles (past $N$ hours) and deduplicate using URL hash & semantic similarity.
### 2. NER & Geocoding / Spatio-Temporal Extraction
- Extract geographic references from article headlines & body text (neighborhoods like *Palermo, Recoleta, Monserrat*, intersections, municipalities, provinces, landmarks).
- Geocode identified locations to coordinates $[lat, lon]$ and Geohashes using OpenStreetMap/Nominatim or local Argentine gazetteer.
- Extract event timestamps (publishing time or mentioned event occurrence time).
### 3. Canonical Formatting & Ed25519 Signing
- Structure into standard Handshake graffiti payload:
```text
📰 {Headline}
{Short Summary / Key takeaway}
📍 {Resolved Location / Neighborhood, Province}
🔗 {Source URL}
```
- Sign payloads canonically with an identified Agent/Bot Ed25519 keypair.
- Attach metadata: tags (`#noticias`, `#buenosaires`, `#politica`, `#cultura`, `#sociedad`), timestamp, and geohash.
### 4. P2P Swarm Broadcasting & Storage
- Ingest into DuckDB `graffitis` store.
- Announce and gossip `.msg` payloads to relevant spatial swarms/topics across the Handshake network.
- Implement decay / TTL policies for ephemeral news vs. long-term historical items.
---
## Tasks
- [ ] Design scraper module with support for Argentine news RSS & HTML parsers (`src/agents/news_agent/crawler.py` or Go equivalent).
- [ ] Implement Geocoding & NER pipeline to resolve Argentine addresses, landmarks, and localities to $[lat, lon]$ + Geohash.
- [ ] Implement content deduplication and rate-limiting to prevent map spam.
- [ ] Integrate Ed25519 canonical message signing with dedicated Agent keypair.
- [ ] Connect agent output to Handshake P2P gossip layer / DuckDB storage engine.
- [ ] Add configurable scheduling / cron runner for autonomous continuous operation.
- [ ] Verify map UI visualization with distinct news markers/badges.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by inspecting the existing agent and Handshake storage or gossip entry points, then determine whether src/agents/news_agent/crawler.py exists. The issue proposes a full crawler, extraction, signing, storage, scheduling, and map-UI pipeline, so the first task is to break it into scoped work. Done would require the complete continuously running pipeline and its UI integration, but no acceptance tests or narrower milestone are specified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data, distributed-systems, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100