Automattic / Automattic/jetpack

Search: Display indexing time to admin when indexing in progress

Open
#15,383 12 comments 0 reactions 0 assignees View on GitHub
[Feature] Search [Pri] Low [Status] Auto-allocated Enhancement Triaged
Dominant language
PHP
Stars
1.8k
Forks
898
Avg merge
1d 18h
Merged PRs (30d)
774

Description

I spun up a jurassic.ninja site that was pre-populated with content and as a part of the set up I immediately purchased a search plan. This is just about the worst case for not allowing Jetpack to sync any data before a user has purchased search and is starting to run searches.

After the purchase I went to the customizer. The search customization works fine, but:
- there are no filters displayed because there are no results
- there are no results no matter what I search for

Looking on the backend I can see that the ES index was not populated for about 5 minutes (we have a 5 minute deduping queue for bulk jobs). Once it was populated the searches I ran before still were not showing any results because they had been cached in the browser. After reloading the browser they still were not shown because we have a five minute timeout on the search query caching if the hash of the search is identical to a previous one.

End result, the user can currently have to wait 10 minutes from after they purchase to when they can first see search results.

Ideas for changes:
- Change our messaging at the end of the purchase flow
- Add a message to the search UI for admins to see how long indexing is expected to take. Let them know that they only have partial results. The API should be able to provide this info.
- ~~Reduce the dedupe queue time? I took a look at doing this, but it is risky and not easy. It also won't solve the whole problem.~~
- ~~Reduce the query caching from 5 min to 2 minutes~~
- ~~Start the indexing as soon as we know the user may be interested in the product~~

cc @keoshi

Contributor guide

Open the contributing guide

Research direction

Start by tracing the search customization UI and the API that exposes indexing state; the issue identifies the Elasticsearch index, purchase flow, and query caching as relevant areas. Determine how admins can be told that indexing is in progress and that results are partial, then verify the message through the purchase-to-search flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch
Domain
api, backend, frontend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.