block / block/elasticgraph

Improve efficiency of `DatastoreIndexingRouter#source_event_versions_in_index`: leverage routing/timestamp

Open
#1,077 0 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Ruby
Stars
88
Forks
36
Avg merge
5d 10h
Merged PRs (30d)
18

Description

This method searches all shards an indices:

https://github.com/block/elasticgraph/blob/89b54463d1af5c0afa00aa8920b0a016c6a758b1/elasticgraph-indexer/lib/elastic_graph/indexer/datastore_indexing_router.rb#L173-L180

This is quite inefficient, particularly when you have many malformed events--it can cause significant load problems on a cluster.

The rationale given in that comment is speculative and I suspect we've never actually had malformed documents with the wrong routing/index rollover timestamp. We should make it more efficient.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.