get-convex / get-convex/geospatial
Broad rectangle queries can exhaust Convex read limits with a few thousand points
- Dominant language
- TypeScript
- Stars
- 29
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
With `@convex-dev/geospatial@0.2.1`, rectangle queries covering a large part of the globe can exhaust Convex's 4,096 index-range read limit with only a few thousand stored points. This makes a normal low-zoom map viewport fail before the application can cluster the returned coordinates.
## Measured reproduction
- Dataset: approximately 3,150 points, spread globally and using eight `filterKeys.table` values.
- Exact world rectangle (`west: -180, south: -90, east: 180, north: 90`): fails with `Too many reads in a single function execution` / `too many system operations`.
- A 180 x 90 degree northwest rectangle: times out after roughly 17 seconds.
- A 180 x 90 degree northeast rectangle: times out after roughly 17 seconds.
- Splitting the world into eight 90 x 90 degree rectangles allows every tile to complete independently, although the two densest tiles still take approximately 12-14 seconds when the caller also resolves metadata per result.
An application-side workaround is to split broad bounds into eight tiles, page each tile under one global point cap, merge and deduplicate the results, and then cluster. However, the component queries' aggregate index work can still approach the execution limit, so this has little safety margin as the dataset grows.
## Additional amplification
`query()` returns `key` and `coordinates`, while the stored point document also contains `filterKeys` and `sortKey`. A consumer that needs the matching `filterKeys` value in the response must call `get()` for every result, causing an N+1 component-query pattern. Returning the stored metadata (perhaps behind an option) would remove that amplification.
## Requested direction
1. A component-native broad rectangle query that bounds/paginates internal index work across smaller coverings, and exposes whether the result is partial?
2. An option for `query()` to include the stored `filterKeys`/`sortKey` fields in each result?
3. Both
Contributor guide
Research direction
Start by tracing the query() entry point and its rectangle index-range work, then inspect how per-result get() calls create the reported N+1 amplification. Use the measured global and large-rectangle reproductions as validation; done requires a defined approach for broad-query limits and a decision on optional filterKeys/sortKey results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, performance, search
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100