php / php/web-php

Search: Index Caching Issues

Open
#1,593 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
1.1k
Forks
640
Avg merge
22h 36m
Merged PRs (30d)
16

Description

The way the search caches indexes is, in my opinion, not ideal.

Background

The current search feature caches the indexes in localStorage, using a unique key for each language.

Related code: https://github.com/php/web-php/blob/e7cfbe98203d93e00aac7c0d159d7c9c3d4faadd/js/search.js#L19 (the first 4 functions, but mostly the first 2, here deal with loading and caching the indexes)

The search indexes are ~2.1 MB (.json file size) and localStorage quota is (generally) 5MB according to https://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria

The current search uses localStorage and caches the indexes with a key for each specific language.

This means that if a user uses the manual in multiple languages, they can store at most 2 before hitting the localStorage quota.

This doesn't stop search working in further languages - the indexes of further visited languages just don't get cached.

However, if a user visits a language, then never returns to it, the indexes for that language are never considered for deletion, and data in localStorage never expires otherwise (unless the user manually clears their data, or the browser decides to delete data from a site not recently visited - I don't know the specifics of browser behavior in this regard)

This means that if the user regularly uses php.net, that (old) unused data remains in localStorage forever, preventing the storage of further data. At the moment, based on a quick search of the web-php repo, this is just other search indexes, but could be used for other functionality in the future.

Alternative Options

I'm not very familiar with explicitly caching data client-side, but based on some (relatively quick) research I think that if this functionality is to be retained, it should be rewritten to use either Cache API or OPFS. These seem to be designed to do exactly what the search is using localStorage for, and the storage quotas are much larger.

An advantage of going with OPFS would be that it would be trivial to manage cached indexes for all languages (listing them in a directory) and deleting unused ones, if that's desirable.

The other alternative would be to get rid of JS caching altogether and just let the browser handle caching as with any other web request.

Does anyone else have thoughts on this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read js/search.js, especially the first four functions and the first two that load and cache language indexes in localStorage. Compare the current per-language caching behavior with the Cache API, OPFS, or browser-managed caching options described in the issue; completion depends on agreeing on and implementing one replacement strategy that handles unused indexes.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.