Automattic / Automattic/liveblog

Remove WPCOM_Liveblog_Entry_Query::get_all_entries_asc()

Open
#591 0 comments 1 reaction 0 assignees View on GitHub
Topic: performance
Dominant language
PHP
Stars
313
Forks
129
Avg merge
18h 33m
Merged PRs (30d)
1

Description

This function is used in a few spots to (primarily) get all entries and then slice out the desired ones in `WPCOM_Liveblog::get_entries_by_time()`, and then to calculate pagination in that function and `WPCOM_Liveblog::get_entries_paged()`.

Normally this works fine, querying the list from memcached then slicing out relevant ones for a given timestamp interval is a fast operation. However, this falls over flat on large datasets. The default memcache max object size is 1MB, and on Liveblog posts with many entries, that can be exceeded, which ends up reading lots of data at once from the DB, as well as writing lots of data to memcache (which is immediately dropped).

`get_comments()` now has Date Query support, and it has some caching internally (although admittedly it's not great b/c the cache key uses the latest comment from the whole site).

To solve the large LB problem, we need to move away from this monolithic cache structure and query the DB, then add caching that only stores small chunks of data.

Contributor guide

Open the contributing guide

Research direction

Start by reading WPCOM_Liveblog_Entry_Query::get_all_entries_asc(), WPCOM_Liveblog::get_entries_by_time(), WPCOM_Liveblog::get_entries_paged(), and get_comments(). Trace how entry lists are read from the database and memcached, then define a chunked query and caching approach that avoids the 1MB object limit while preserving timestamp filtering and pagination.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.