Denormalize base_type onto identifier table for query performance
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Description
We can improve our db lookup/query performance when querying the db for identifiers of a specific base type like navigation or content drive.
This will be done in 2 PRs. The first does the data model changes, the second changes the SQL we use to query. The data model changes will be in code for at least a month before we merge the code that does the query changes.
Acceptance Criteria
- startup task that add columns, index and indexes
- remove unneeded column joins
- Pass tests and don't break starter loads or push publishing bundes
- 2 prs
Priority
None
Additional Context
We have data that shows on a loaded site, these type of queries are some of the heaviest - here are the top 3 querys for total_exec_time for one customer
query | select cvi.working_inode as inode from contentlet_version_info cvi, identifier id, structure struc, contentlet c where cvi.identifier = id.id and struc.velocity_var_name = id.asset_subtype and c.inode = cvi.working_inode and cvi.variant_id=$3 and struc.structuretype in ($4 , $5) and (id.host_inode = $1) and id.parent_path=$2 and c.show_on_menu = $6 and cvi.deleted = $7 order by c.mod_date asc
calls | 22387276
total_exec_time | 3649630092.113903
mean_exec_time | 163.02251743866006
-[ RECORD 2 ]---+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
query | SELECT * FROM QRTZ_EXCL_LOCKS WHERE LOCK_NAME = $1 FOR UPDATE
calls | 5382189
total_exec_time | 3519143867.950299
mean_exec_time | 653.8499238786048
-[ RECORD 3 ]---+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
query | select cvi.live_inode as inode from contentlet_version_info cvi, identifier id, structure struc, contentlet c where cvi.identifier = id.id and struc.velocity_var_name = id.asset_subtype and c.inode = cvi.live_inode and cvi.variant_id='DEFAULT' and struc.structuretype in (4) and (id.host_inode = $1) and id.parent_path=$2 and cvi.deleted = 'false' order by c.mod_date desc
calls | 23572980
total_exec_time | 1129625881.7285948
mean_exec_time | 47.92036822364463
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the identifier table and data model, then compare them with the three example queries in the issue. Define the startup task for the new columns and indexes, verify the data-model and query changes can be split into two PRs, and run the tests plus starter-load and push-publishing checks to confirm the work is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100