developmentseed / developmentseed/gazet
Tracking: places that should resolve but return a wrong fuzzy match
- Dominant language
- Python
- Stars
- 23
- Forks
- 2
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
## What happens
A well-known city can be absent from the index, and `/search` answers
anyway — with a fuzzy match on a different place, often on another
continent. There is no "no match", and `similarity` does not mark it as
weak, so a caller taking the top result builds a confident wrong answer.
```console
$ curl -sL 'https://developmentseed-gazet.hf.space/api/search?mode=fuzzy&q=Manaus&ids_only=true&limit=5' \
| jq -r '.ids[] | "\(.similarity | .*1000 | round / 1000) \(.name) (\(.subtype), \(.country // "—"))"'
0.967 Manas (county, KG)
0.967 Manas (river, —)
0.933 Manus (region, PG)
0.933 Manus Island (island, —)
0.933 Mana (county, GF)
```
Manaus, Brazil — population ~2.2M — is not among them. The top hit is a
county in Kyrgyzstan, **13,719 km away**, returned at `similarity`
0.967.
Qualifying the query makes it worse:
```console
$ curl -sL '.../api/search?mode=fuzzy&q=Manaus,%20Brazil&ids_only=true&limit=3' | jq -r '.ids[].name'
Manas
Manas
Managua Department
```
## `similarity` does not separate right from wrong
Twenty cities through `/search`, taking the top result and measuring its
bbox centre against the city's real position:
| Query | Top candidate | `similarity` | km off |
|---|---|---|---|
| Manaus | Manas | **0.967** | 13,719 |
| Porto Alegre | Portalegre | **0.967** | 8,922 |
| Chittagong | Chitato | 0.911 | 8,440 |
| Bogota | Bobota | 0.911 | 10,132 |
| Da Nang | Na Duang | 0.911 | 680 |
| Fortaleza | Fortul | 0.878 | 3,871 |
| Recife | Partido de Arrecifes | 0.539 | 3,866 |
| Casablanca | Casablanca-Settat | 0.918 | 64 ✅ |
| Antananarivo | Antananarivo Avaradrano | 0.904 | 13 ✅ |
| Marrakesh | Marrakesh Prefecture | 0.890 | 12 ✅ |
| Hobart | City of Hobart | **0.325** | 4 ✅ |
**Wrong results score 0.539–0.967. Correct results score 0.325–1.000.**
The ranges overlap almost entirely, and the extremes invert: the single
lowest-scoring result in the set is correct, while the two highest below
1.0 are wrong by thousands of kilometres. No threshold separates them,
because the score measures string distance between names and the failure
is a near-homograph in another country.
One usable signal does fall out: **every result scoring exactly 1.000 was
correct** (Nairobi, Lisbon, Reykjavik, Almaty, Kampala, Kolkata, Chennai,
Surabaya, Ulaanbaatar — 9 of 9). It is only below an exact match that the
score stops meaning anything.
## It is two problems, not one
### 1. Present in the index, unreachable by the query
Three of the seven are there, under their local or official name:
| Query | Returns | But the index does contain |
|---|---|---|
| Bogota | `Bobota` (RO) | `Bogotá, D.c.` (CO, county) — found by `q=Bogotá` at 0.908 |
| Chittagong | `Chitato` (AO) | `Chattogram District` (BD, county) — found by `q=Chattogram` |
| Da Nang | `Na Duang` (LA) | `Đà Nẵng` (VN, region) — found by `q=Đà Nẵng` at 1.000 |
Jaro-Winkler is prefix-weighted, so an unaccented English exonym scores a
near-homograph on another continent above the real place, which is
spelled differently. `matched_name` equals `name` on every result I saw,
so nothing is matching on an alternate spelling.
### 2. Pulled, then discarded at ingest
The four remaining are Brazilian cities, and they are **not missing from
Overture** — the README syncs the whole `type=division_area`, so they are
downloaded every time. They are dropped one line into normalisation:
```sql
-- dataset/scripts/normalize_geodata.py:56
AND subtype IN ('country', 'region', 'county')
```
Against the 2026-08-19.0 release that keeps **43,046 of 4,658,700 rows —
99.1% discarded**, and cities are in the part thrown away:
| subtype | rows (`type=division`) | kept today |
|---|---:|---:|
| locality | 3,481,755 | 0 |
| neighborhood | 721,822 | 0 |
| microhood | 235,159 | 0 |
| macrohood | 155,479 | 0 |
| **county** | **38,908** | **38,908** |
| localadmin | 21,386 | 0 |
| **region** | **3,919** | **3,919** |
| **country** | **219** | **219** |
**`county` is not a universal level, and that is why the gap looks
arbitrary.** Whether a country's cities survive the filter depends on how
its administrative hierarchy happens to map onto Overture's subtypes:
| country | `county` rows | `locality` rows |
|---|---:|---:|
| Brazil | **0** | 84,747 |
| Vietnam | **0** | 16,498 |
| Bangladesh | 64 | 5,371 |
| Kenya | 291 | 10,601 |
| Australia | 599 | 16,406 |
| Romania | 3,179 | 14,414 |
Brazil and Vietnam have no county rows at all, so every Brazilian and
Vietnamese city is dropped. Kenya and Australia do, which is the whole
reason `Nairobi County` and `City of Hobart` resolve.
Manaus is present, as an area, with the right geometry:
```console
name subtype region population xmin ymin
Fortaleza locality BR-CE 2428708 -38.64 -3.89
Manaus locality BR-AM 2063689 -60.80 -3.18
Recife locality BR-PE 1488920 -35.02 -8.16
Porto Alegre locality BR-RS 1332833 -51.30 -30.27
```
## Enriching without adding a new source
1. **Widen the subtype filter** at `normalize_geodata.py:56` to include
`localadmin` and `locality`. In `division_area` — the type actually
synced — that is 553,493 localities and 21,386 localadmin, so the
extract goes from ~43k to ~615k rows, not the 3.5M the point table
suggests. This alone puts Manaus, Recife, Fortaleza and Porto Alegre
in the index.
2. **Rank by population, which is one join away.** `division_area` has no
`population` column, but it carries `division_id`, and
`type=division` (a single 0.6 GB file in the same release) has
`population`, `wikidata` and `hierarchies`. Joining it makes the
headline failure sort itself out:
```
name subtype country population
Manaus locality BR 2063689
Manas locality IN 7000
Manas locality FR 183
Manas locality TR NULL
```
Treat it as a boost, not a filter — `population` is set on only 17.8%
of localities, so a null must not outrank an exact name match.
Ranking matters more once localities are in: `Recife` matches 14
Brazilian localities across eight states and `Fortaleza` 88. Adding
the rows without a prominence signal trades a wrong-continent answer
for a wrong-village one.
3. **Index Overture's alternate names.** Division records carry
`names.common` (language → name) and `names.rules` beside
`names.primary`. Indexing those resolves Bogota, Chittagong and Da
Nang from data already in the extract — and the response already has a
`matched_name` field to report which spelling hit.
4. **Fold diacritics on both sides** (NFKD, drop combining marks) so
`Bogota` reaches `Bogotá` even where no alternate name exists.
Nothing here needs a source that is not already downloaded.
## What this issue is for
A running list of places that should resolve and don't, so they can be
added as they are found. Seven above to start — Manaus, Porto Alegre,
Bogota, Chittagong, Fortaleza, Recife, Da Nang. **Please add to this
issue rather than opening one per city.**
Beyond the individual entries, **return no match when nothing is
genuinely close** — or a confidence that reflects being on the wrong
continent rather than the edit distance between two names. Right now the
response shape for "Manaus" is indistinguishable from the one for
"Nairobi".
## Reproducing
```bash
curl -sL 'https://developmentseed-gazet.hf.space/api/search?mode=fuzzy&q=Manaus&ids_only=true&limit=5' | jq '.ids'
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure with the provided curl command, then read dataset/scripts/normalize_geodata.py around line 56 and trace how its output reaches the /api/search endpoint. Compare the proposed subtype, population, alternate-name, diacritic, and no-match requirements with the current index and response behavior. Done means the listed places resolve appropriately without confident wrong-continent matches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- backend-api-design, data-engineering, search
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100