ome / ome/openmicroscopy

Improve performance of graph operations.

Open
#6,243 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
222
Forks
105
Avg merge
2h 21m
Merged PRs (30d)
1

Description

Three separate sources of slowness have been identified:

  1. Early in the process when data is being gathered, profiling shows Hibernate eating a lot of memory. Caches may need configuration and later versions of Hibernate may make that easier. GraphTraversal will tend not to be querying the same objects repeatedly.

  2. In reviewing the data gathered via Hibernate, GraphPolicyRule eats much CPU time in rule-matching. The internal implementation of the class could do non-trivial static analysis when initialized, compiling the list of rules to a decision tree of checks, branches, changes, making the subsequent matching much faster.

  3. In actioning the model changes, PostgreSQL can spend much time in managing the _reindexing_required table. Perhaps migration to https://github.com/glencoesoftware/omero-es can somehow help.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by profiling the three named areas: Hibernate during data gathering, GraphPolicyRule during rule matching, and PostgreSQL work involving the _reindexing_required table. Establish separate benchmarks and narrow the issue to a concrete bottleneck before changing implementation; done should mean a measured performance improvement in the affected graph operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
backend, databases, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.