locationtech / locationtech/geomesa
Optimize trino count queries that include visibility checks
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 446
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 31
Description
Right now, filtering by visibilities requires scanning all rows. This slows down queries which should be faster, such as querying for counts. The solution might involve using file metadata to filter out entire files whose visibility values are higher that that of the user performing the query. Other solutions might also exist and should be explored for effectiveness.
Constraint: A solution should not require re-writing of existing data. Updating metadata, however, is okay.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating GeoMesa's visibility filtering and count-query planning, then inspect how file metadata is available during scans. Compare candidate approaches with count-query benchmarks, ensuring existing data does not need rewriting. Done means visibility-aware counts avoid scanning rows that can be excluded and remain correct for different user visibility levels.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- databases, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100