INCATools / INCATools/ontology-access-kit
Performance: `SqlImplementation.entity_metadata_map()`
- Dominant language
- Python
- Stars
- 198
- Forks
- 35
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 1
Description
## Overview
This method seems far too slow.
## Example
In [this PR](https://github.com/monarch-initiative/mondo-ingest/pull/363#discussion_r1328031211), using `mondo.db`:
```py
exclusion_rels: List[Tuple[str, str]] = []
for rel in oi.entities_metadata_statements(mondo_ids, ['obo:mondo#excluded_subClassOf']):
if rel:
exclusion_rels.append((rel[0], rel[2]))
```
Each iteration of this loop took on average 15.5 seconds. Estimating that to get through ~25k mondo_ids would take ~100 hours at this rate.
Contributor guide
Research direction
Start with SqlImplementation.entity_metadata_map() and reproduce the example query against mondo.db, measuring the roughly 15.5-second iterations described in the issue. Trace the method's work to identify the source of the slowdown. Done means the example completes substantially faster than the reported estimate for about 25k mondo_ids.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100