openwatersio / openwatersio/tide-database

Add IOC Sea Level Station Monitoring Facility as a derived source (fit constituents from QC'd observations)

Open
#124 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
25
Forks
6
Avg merge
3d 6h
Merged PRs (30d)
10

Description

Summary

The IOC Sea Level Station Monitoring Facility (SLSMF, run by VLIZ for UNESCO/IOC) relays ~1,600 real-time tide gauges worldwide. It publishes raw relative sea level only — no harmonic constituents, no vertical datum — which is why none of it is in this database today. Many of its gauges do arrive indirectly through GESLA-4 → TICON-4, but a large set never reach GESLA, mostly newer tsunami-warning gauges.

This proposes IOC as a derived source: download quality-controlled observations from their REST API, fit constituents with the fitter that already lives in tools/harmonic-analysis.ts, and reduce datums with computeDatumsFromObservations. It would be the first source with published_harmonics: false.

Coverage gain

Comparing the IOC station list (excluding DART buoys) against the current stations export:

Count
IOC gauges with coordinates 1,592
Reporting in the last 30 days 1,106
Within 3 km of an existing station 1,031
Not covered and active 343

The uncovered active gauges cluster in Chile (77), Taiwan (44), Indonesia (22), Greece (16), Russia (15), Peru (12), Brazil (11), Italy, Oman, Norway, Turkey, Spain, Colombia, Australia, Cyprus, Malaysia, plus scattered Caribbean and Pacific island gauges. Real yield will be lower once the ≥365-day record gate and QC completeness flags are applied.

Data access

Two APIs:

  • v1 service.php — unauthenticated, raw 1-minute levels, ~30 days per request, no QC. Works today but returns junk unfiltered.

  • v2 REST (https://api.ioc-sealevelmonitoring.org/v2/) — requires an X-API-KEY (free account, then request the "gauges API" group). The relevant route is the research endpoint, public since March 2025:

    GET /v2/research/stations/{code}/sensors/one-sensor/data
        ?timestart=YYYY-MM-DD&timestop=YYYY-MM-DD&days_per_page=365&page=N
    

    Returns QC'd data, max 365 days per request, with daily flags (completeness, distinctness, shift) and per-sample filters (out of range, spikes via median, flat line, exceeded neighbours). Flagged samples are set to NA rather than dropped, which is what the least-squares fit wants. fit_to_sample_rate=true aligns timestamps to the sensor rate. OpenAPI spec: https://api.ioc-sealevelmonitoring.org/v2/doc/json.

Recommend v2 research. The shift flag in particular does the datum-jump detection we currently have to eyeball.

What the importer needs

  1. Fetcher — page the research endpoint a year at a time per station, convert to the existing Sample[] shape ({ t: epoch ms, level: m }). Cache under tmp/ like download-gesla.ts.
  2. Selection — only IOC gauges farther than ~3 km from an existing station, with ≥365 days of QC'd data (isAnalyzable), reporting recently.
  3. FitfitHarmonics + computeDatumsFromObservations, same path import-ticon.ts uses for the WSV re-fit and observed datums.
  4. Station JSON — new source block (name: "IOC SLSMF", url, id: <ioc code>, published_harmonics: false), datums_source: "observed", and a disclaimer that datums are in the gauge's own relative frame.

Open questions

  • Licence. IOC states no licence, only a citation (Flanders Marine Institute (VLIZ); IOC (2026), DOI 10.14284/482) and a "no QC, no liability" disclaimer. Should confirm with VLIZ that redistributing derived constituents is acceptable, and decide what the license block says. Relevant to #52.
  • Datum. IOC explicitly holds relative levels only: "We only deal with relative data, we don't have the vertical datum information available." Every IOC-derived station would carry observed datums in the gauge's frame, like the TICON harmonic fallbacks do now. Fine for timing and range, weak for chart-datum heights. See docs/datums.md and #105.
  • Chain of custody (#53) — the source.id should be the IOC code; the SLSMF catalog maps those to UHSLC/PSMSL/GLOSS ids where they exist.

Motivating example

Cane Garden, Tortola, BVI (IOC tbvi2, WMO SOVI10). Nearest database stations are NOAA St. John gauges ~10 km away. It is not a first-pass candidate: the record starts mid-2024, has a datum jump around early 2025, and the radar has been emitting 0/1/~30 values since roughly November 2025. It needs a repair and a fresh year of data. But it is representative of the newer Caribbean and Pacific gauges this source would pick up.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/harmonic-analysis.ts, computeDatumsFromObservations, import-ticon.ts, and download-gesla.ts to understand the existing fitting, datum, and caching paths. Review docs/datums.md and the IOC v2 research API details, then define the fetch, station-selection, source metadata, licensing, and relative-datum behavior; done means a reproducible importer producing eligible IOC-derived station JSON.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.