apache / apache/sedona-spatialbench

Commit executed outputs for the geography query notebook

Open
#133 0 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Rust
Stars
48
Forks
23
Avg merge
1d 50m
Merged PRs (30d)
4

Description

Follow-up to #131, which adds the geography query suite.

`notebooks/queries.ipynb` ships **with** executed outputs — every query cell has a result table, and those tables are what appear in `docs/queries.md`. `notebooks/geography-queries.ipynb` was committed **without** outputs, because executing it today would put misleading results into the docs.

## Why it was deferred

Two SedonaDB defects, both filed and both engine-side:

- **apache/sedona-db#1084** — a geography scalar function over the output of a geography spatial join fails with `Array input must not be empty`. Q9 raises this, so its cell would carry an error traceback.
- **apache/sedona-db#1085** — one zone polygon of 156 095 (`z_zonekey` 150480) returns a negative geography `ST_Area` and matches points anywhere on Earth. Q4, Q6, Q10 and Q11 are all distorted by it: at SF1, Q6 would display `Shendi | 5999980` — essentially every trip in the dataset, for a county nowhere near the query area — and Q11 would report 6 290 682 cross-zone trips against the planar suite's 176 391.

Committing those outputs would document engine bugs as if they were the queries' results, and the page deliberately carries no engine-defect discussion.

Executing only the sound subset was considered and rejected: a notebook where some cells have outputs and others do not reads as broken rather than deliberate.

## What to do

Once #1084 and #1085 are released:

1. Install the docs environment (`docs/requirements.txt`, which brings `apache-sedona[db]`) and generate or download SF1 data.
2. Run `notebooks/geography-queries.ipynb` top to bottom. Q12 is a markdown cell, not a code cell, so it will not error — it needs a geography `ST_KNN` (apache/sedona-db#1086, apache/sedona#3178).
3. Regenerate the page: `python -m nbconvert --to markdown notebooks/geography-queries.ipynb`, then re-prepend the YAML front matter and ASF license header that `docs/geography-queries.md` carries (the notebook itself is RAT-excluded via `*.ipynb`; the page is not, unlike `docs/queries.md`).
4. Run `pre-commit run --files ...` — the end-of-file fixer adjusts nbconvert's trailing newlines.
5. Mirror the new output blocks into `docs/geography-queries.zh.md`, which is maintained by hand from the English page.

## Sanity checks worth keeping

At SF1 the geography results were verified against the committed geometry answers: Q2 matches exactly (541), Q3 matches its row count (22), Q4 reproduces the geometry answer row for row (258 zones, same counts, same order) once the #1085 polygon is excluded, and Q9's self-join finds the same 37 building pairs. Those are good regression anchors when the outputs are finally committed.

Contributor guide

Open the contributing guide

Research direction

Wait for apache/sedona-db#1084 and #1085 to be released, then install docs/requirements.txt and generate or download SF1 data. Run notebooks/geography-queries.ipynb top to bottom, regenerate docs/geography-queries.md with its existing front matter and ASF header, run pre-commit on the changed files, and mirror the output blocks into docs/geography-queries.zh.md. Done means the documented outputs pass the listed Q2, Q3, Q4, and Q9 sanity checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, sql
Domain
databases, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.