databrickslabs / databrickslabs/ontos
perf: batch domain lookups in full search-index reindex (follow-up to #520)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Follow-up from PR #597 (multi-domain assignment, #520).
Context
The full search-index reindex path (get_search_index_items for assets / contracts / products) issues a per-entity domain query when building each SearchIndexItem. The paged list paths already batch via EntityDomainAssociationRepository.get_domains_for_entities; the incremental/single-entity reindex fallback does not.
Impact
Performance only, on an infrequent path (full reindex is a background/admin operation, not a hot request). No correctness issue.
Suggested fix
Batch-load domains once per entity type via get_domains_for_entities and pass them into _build_search_index_item (a preloaded_domains param already exists for products — extend the same pattern to the reindex loop for the other entity types / retain the single-entity fallback only for genuinely incremental updates).
Non-blocking; surfaced during code review of #597.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at get_search_index_items for assets, contracts, and products, then trace _build_search_index_item and EntityDomainAssociationRepository.get_domains_for_entities. Compare the existing products preloaded_domains path with the full reindex loops and retain the single-entity fallback for incremental updates. Done means full reindexing batches domain lookups without changing search-index results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance, search
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100