Improve efficiency of `DatastoreIndexingRouter#source_event_versions_in_index`: leverage routing/timestamp
Open
enhancement
good first issue
- Dominant language
- Ruby
- Stars
- 88
- Forks
- 36
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 18
Description
This method searches all shards an indices:
https://github.com/block/elasticgraph/blob/89b54463d1af5c0afa00aa8920b0a016c6a758b1/elasticgraph-indexer/lib/elastic_graph/indexer/datastore_indexing_router.rb#L173-L180
This is quite inefficient, particularly when you have many malformed events--it can cause significant load problems on a cluster.
The rationale given in that comment is speculative and I suspect we've never actually had malformed documents with the wrong routing/index rollover timestamp. We should make it more efficient.
Contributor guide
Assessment
This issue has not been assessed yet.