Improve performance of the History page queries
Nobody has claimed this yet.
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
Coming off #968.
We have several places where performance of the History page query is hampered, here is a list of things I noticed:
- We should not cast columns when searching, for example both
log_linesand thedatacliptables have their columns cast into varchars - this is very expensive. - Finding the last run by joining on
finished_atis exponentially less efficient depending on how many attempts & runs there are.
Proposed solutions
- #1898
- #1899
- Add index to
attempt_runsforattempt_idand/orrun_id, currently the compound index doesn't appear to be used for these queries.
These solutions are distinct (and should be done regardless) from taking another approach which is creating aggregate tables
- Create aggregate (or columns) that get updated when an attempt/run is created or updated. Removing the need to join and filter Runs by exit code and finished_at.
References:
invocation.ex:417 - list_work_orders_for_project_query/2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at invocation.ex:417 and trace list_work_orders_for_project_query/2, then review the remaining unchecked work in #1899 and the proposed attempt_runs index. Done means the History page queries avoid the listed column casts and inefficient finished_at joins, with the query behavior and performance improvements verified against the stated cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, databases, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100