Module proposal: Cache `alloptions` to load only needed values

This proposes a module for the Performance Lab Plugin.

Overview

• Proposed module name: Conditional Option Autoloading:
• Proposed module slug: conditional_option_autoloading
• Proposed module owner GitHub username(s): pbearne

About the module

Early in the WordPress load, we load from the wp_options all the rows with the column autoload set "yes" this causes un-needed data to be stored in memory via the wp_load_alloptions function

The default value for autoloading is yes so unless it was a careful/knowledgable dev the opinion will set to autoload

This problem is increased as unless there is a deactivation function in a plugin then orphan rows(values) will still be loaded for plugins that have been removed.

This proposal is that only load the needed rows(values) for a given page

Purpose

By listening to calls get_option and recording the keys that are used for a current URL and are logged in
Then from keys get the ID for rows and save these IDs on shutdown to site cache (hopefully into Redis/Memcache)
Then on future calls to the same page, we use the keys to shortcut the get_all_options function to just the keys needed for that page.

To invalidate the cached ids we keep listening to every page in the get_option function to create a list of option names call for and then compare it to the list of options names for values we preloaded in get_all_options and if it has changed we save the new list of IDs

Only logged out for start until we have worked the best way to handle login and user ID
But it will be sites with logged users that may benefit the most
So we may need to look user ID in the key name and not add that key in case the plugin is storing user data in options

This will need a set of filters in WP core to enable this

• pre_option_all https://core.trac.wordpress.org/ticket/37930
• pre_get_alloptions https://core.trac.wordpress.org/ticket/56045

And we may need to load the code in sunrise.php or via Object-cache.php so we can run the code before wp_load_alloptions is called

Scope

We are not creating a caching service just calling the installed service (if any)
We are not storing any option values just the IDs needed for the given page

Rationale

This issue may be described as a technical debt as it has resulted from the default setting being set to autoload to true
So we are trying to reduce the size of the options data stored in mem for every page and not used

Other

Issues

If there are option names that have a user identifier included these will cause the cache to invalidate each use
to partly mitigate this we may need to look for the user-id in the option_name
we will need real-world testing to see if this is a problem

this will increase the size of the cache by one item per page

Some starter/proof of concept code
https://github.com/pbearne/wordpress-develop/tree/conditional_options_preload