get-convex / get-convex/geospatial

Broad rectangle queries can exhaust Convex read limits with a few thousand points

Open
#57 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
29
Forks
6
PR merge metrics
No merged PRs in 30d

Description

## Summary

With `@convex-dev/geospatial@0.2.1`, rectangle queries covering a large part of the globe can exhaust Convex's 4,096 index-range read limit with only a few thousand stored points. This makes a normal low-zoom map viewport fail before the application can cluster the returned coordinates.

## Measured reproduction

- Dataset: approximately 3,150 points, spread globally and using eight `filterKeys.table` values.
- Exact world rectangle (`west: -180, south: -90, east: 180, north: 90`): fails with `Too many reads in a single function execution` / `too many system operations`.
- A 180 x 90 degree northwest rectangle: times out after roughly 17 seconds.
- A 180 x 90 degree northeast rectangle: times out after roughly 17 seconds.
- Splitting the world into eight 90 x 90 degree rectangles allows every tile to complete independently, although the two densest tiles still take approximately 12-14 seconds when the caller also resolves metadata per result.

An application-side workaround is to split broad bounds into eight tiles, page each tile under one global point cap, merge and deduplicate the results, and then cluster. However, the component queries' aggregate index work can still approach the execution limit, so this has little safety margin as the dataset grows.

## Additional amplification

`query()` returns `key` and `coordinates`, while the stored point document also contains `filterKeys` and `sortKey`. A consumer that needs the matching `filterKeys` value in the response must call `get()` for every result, causing an N+1 component-query pattern. Returning the stored metadata (perhaps behind an option) would remove that amplification.

## Requested direction

1. A component-native broad rectangle query that bounds/paginates internal index work across smaller coverings, and exposes whether the result is partial?
2. An option for `query()` to include the stored `filterKeys`/`sortKey` fields in each result?
3. Both

Contributor guide

Open the contributing guide

Research direction

Start by tracing the query() entry point and its rectangle index-range work, then inspect how per-result get() calls create the reported N+1 amplification. Use the measured global and large-rectangle reproductions as validation; done requires a defined approach for broad-query limits and a decision on optional filterKeys/sortKey results.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, performance, search
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.