cacrespo / cacrespo/handshake

feat(agent): live Argentine news crawler & spatio-temporal graffiti agent

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
2m
Merged PRs (30d)
1

Description

## Description

Once Handshake is live, deploy an autonomous live agent / daemon that continuously crawls and scrapes Argentine news outlets (national & regional), extracts spatial and temporal event context, and signs & publishes them as real-time **geo-temporal graffitis** on the Handshake map.

Users opening Handshake will see live breaking news, local events, traffic/protests, and cultural updates physically anchored to the exact locations in Argentina where the events are happening.

---

## Architecture & Pipeline

### 1. Ingestion & Web Scraping Engine
- Monitor RSS feeds, sitemaps, and live pages of national outlets (*Infobae, Clarín, La Nación, Página/12, Ámbito, Télam/Somos Télam, etc.*) as well as regional Argentine media (*La Voz, El Día, La Capital, etc.*).
- Filter recent articles (past $N$ hours) and deduplicate using URL hash & semantic similarity.

### 2. NER & Geocoding / Spatio-Temporal Extraction
- Extract geographic references from article headlines & body text (neighborhoods like *Palermo, Recoleta, Monserrat*, intersections, municipalities, provinces, landmarks).
- Geocode identified locations to coordinates $[lat, lon]$ and Geohashes using OpenStreetMap/Nominatim or local Argentine gazetteer.
- Extract event timestamps (publishing time or mentioned event occurrence time).

### 3. Canonical Formatting & Ed25519 Signing
- Structure into standard Handshake graffiti payload:
```text
📰 {Headline}
{Short Summary / Key takeaway}
📍 {Resolved Location / Neighborhood, Province}
🔗 {Source URL}
```
- Sign payloads canonically with an identified Agent/Bot Ed25519 keypair.
- Attach metadata: tags (`#noticias`, `#buenosaires`, `#politica`, `#cultura`, `#sociedad`), timestamp, and geohash.

### 4. P2P Swarm Broadcasting & Storage
- Ingest into DuckDB `graffitis` store.
- Announce and gossip `.msg` payloads to relevant spatial swarms/topics across the Handshake network.
- Implement decay / TTL policies for ephemeral news vs. long-term historical items.

---

## Tasks

- [ ] Design scraper module with support for Argentine news RSS & HTML parsers (`src/agents/news_agent/crawler.py` or Go equivalent).
- [ ] Implement Geocoding & NER pipeline to resolve Argentine addresses, landmarks, and localities to $[lat, lon]$ + Geohash.
- [ ] Implement content deduplication and rate-limiting to prevent map spam.
- [ ] Integrate Ed25519 canonical message signing with dedicated Agent keypair.
- [ ] Connect agent output to Handshake P2P gossip layer / DuckDB storage engine.
- [ ] Add configurable scheduling / cron runner for autonomous continuous operation.
- [ ] Verify map UI visualization with distinct news markers/badges.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by inspecting the existing agent and Handshake storage or gossip entry points, then determine whether src/agents/news_agent/crawler.py exists. The issue proposes a full crawler, extraction, signing, storage, scheduling, and map-UI pipeline, so the first task is to break it into scoped work. Done would require the complete continuously running pipeline and its UI integration, but no acceptance tests or narrower milestone are specified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data, distributed-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.