OvertureMaps / OvertureMaps/data

[Places][2026-07-22.0] About 40% of websites values for Türkiye do not respond; a reachability signal would help

Open
#571 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
47
Avg merge
3h 42m
Merged PRs (30d)
1

Description

Summary

Of the Türkiye places that carry a websites value, roughly two in five do not answer an HTTP request. There is nothing in the record that lets a consumer tell a listed website from a working one, so any adoption or digital-presence figure built on the field overstates by about a third. Adding a last-checked timestamp or a reachability flag to the websites entries, or dropping values that have failed for several releases, would fix that at the source.

Release and scope

  • Release: 2026-07-22.0, theme places, type place, Türkiye (bbox 25.5..45.0, 35.7..42.3, addresses[1].country = 'TR', names.primary IS NOT NULL, operating_status not closed)
  • 1,786,700 records in scope, 684,496 with a non-empty websites value (38.3%)

What we measured

We requested the first websites value of every one of those 684,496 records once, in September 2026, from a host in Türkiye:

  • GET on the URL as listed (https:// prefixed if no scheme), redirects followed (max 5), 12 s timeout, identifiable user agent, headers read only, one request per site, no retries.
  • Counted as reachable: a 2xx response.
  • Counted as unreachable: connection or DNS failure, timeout, or a non-2xx status.

Result: 413,777 reachable (60.5%), 270,719 unreachable (39.5%). So the share of Turkish places with a working website is 23.2%, not the 38.3% the field suggests.

Caveats: a single attempt on a single day will misclassify some sites that were briefly down or that block non-browser clients, so the true unreachable share is somewhat lower than 39.5%. It is not close to zero: the pattern by province and sector is stable and the bulk are parked domains, expired hosting and DNS that no longer resolves.

Breakdown

Per-province and per-sector counts (listed vs reachable) are published as CSV/JSON under CC BY 4.0, with the SQL that produced them:

We can share the per-record list (GERS id, URL as listed, failure class) if it is useful for the Places pipeline; it is ~270k rows.

Suggestion

Any of these would let consumers stop re-crawling:

  1. a last_verified (or similar) timestamp per websites entry,
  2. a boolean or status alongside the URL when the pipeline already checks liveness,
  3. or removal of URLs that have failed across consecutive releases.

Related: #570 (the addresses[].region field for the same extract).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Places pipeline and schema definition for the websites field, then inspect how release 2026-07-22.0 produces it. Compare the proposed reachability representations against the reported Turkish cases and determine which the project can support; done means consumers can distinguish listed from reachable websites without re-crawling.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.