Change the term search to find individual words rather than an exact match

Currently a search for multiple words will only match terms whose description (or synonyms) exactly contain the entire search string. A search for leg fracture would only match terms with this exact sequence, so it would not match fracture of leg, fracture of lower leg etc.

This is particularly problematic in coding systems with few or no synonyms. E.g. in ICD10 to find leg fractures you need to first search broadly for fracture before updating your search to fracture of femur and fracture of lower leg once you see the possible fracture concept definitions.

The issue is masked to some extent in SNOMED because each definition has several synonyms, but potentially misleads people into thinking the searching is doing something that it isn't. E.g. the first match when searching for type 2 diabetes is Diabetes mellitus type 2 - because we don't show that the match actually occurred in the synonym Type 2 diabetes mellitus, a user may think the search has in fact matched any description containing type, 2 and diabetes in any order.

This ticket is a proposal (discussed here) to change the search so that multi-word searches are tokenized, and the search results are anything that matches ALL individual words.

Why are we doing this?

• The existing search strategy could be changed to improve the user experience, allowing fewer more targeted search terms
• Concepts missed with current search strategies may be found with the new approach
• We don't want to mislead users as to how the search is implemented

How will we know when it's done?

The new search strategy has been implemented in all coding systems where it is appropriate to do so.

What are we doing?

• Determine a tokenization strategy. Probably just split a search term by white space, but perhaps consider non-alphanumeric characters as well.
• Implement the new search strategy where we match concepts whose definitions contain ALL tokenized words from the search term
• Search strings within quotes should not be tokenized so users can still do exact searches e.g. "blood pressure"
• All existing searches should be changed so they appear in "s via a migration. This ensures that new versions of these codelists maintain the original search intent.

Defining delivery tasks guidance

Start by locating the current term-search implementation and existing saved searches across the coding systems. Resolve tokenization, quoted exact searches, and the migration strategy before implementing matching of all tokens. Done means the appropriate coding systems use the new behavior, quoted searches remain exact, and existing searches retain their original intent.