typesense / typesense/typesense-website

`stopwords` search param documented as a comma separated list, but the engine accepts only a single set name (404s otherwise)

Open Beginner friendly
#503 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Vue
Stars
66
Forks
201
Avg merge
12d 17h
Merged PRs (30d)
4

Description

Summary

The search-parameter table documents stopwords as accepting a comma separated list of stopword sets, but the engine treats the whole value as a single set name. Passing two set names comma-separated fails the entire search with HTTP 404.

Docs (27.1 → 30.2, e.g. 30.2/api/search.html):

stopwords — A comma separated list of stopword sets to be dropped from the search query while searching.

Steps to reproduce

Tested on self-hosted Typesense 30.1 and 30.2.

curl "http://localhost:8108/stopwords/p_a" -X PUT \
  -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
  -d '{"stopwords":["alpha"],"locale":"en"}'

curl "http://localhost:8108/stopwords/p_b" -X PUT \
  -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
  -d '{"stopwords":["beta"],"locale":"en"}'

# one set -> works
curl -G "http://localhost:8108/collections/books/documents/search" \
  -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
  --data-urlencode 'q=alpha widget' --data-urlencode 'query_by=title' \
  --data-urlencode 'stopwords=p_a'

# two sets, as documented -> 404, the whole query fails
curl -G "http://localhost:8108/collections/books/documents/search" \
  -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
  --data-urlencode 'q=alpha beta widget' --data-urlencode 'query_by=title' \
  --data-urlencode 'stopwords=p_a,p_b'

Expected (per the docs): both sets applied — alpha and beta dropped from the query.

Actual:

{"message":"Could not find the stopword set named `p_a,p_b`."}

Same result via multi_search with "stopwords": "p_a,p_b". Passing a JSON array ("stopwords": ["p_a","p_b"]) returns One or more search parameters are malformed.

The comma is part of the name, not a separator

To rule out an escaping/encoding mistake on my side, I created a third set whose name literally contains a comma:

set id words
p_a alpha
p_b beta
p_a,p_b gamma
curl -G ".../documents/search" --data-urlencode 'q=gamma widget' \
  --data-urlencode 'query_by=title' --data-urlencode 'stopwords=p_a,p_b'

gamma was dropped (the query became widget and matched every document), while alpha/beta were untouched — i.e. the engine resolved the set named p_a,p_b, it did not union p_a and p_b. Once such a set exists the request stops 404-ing, which also explains the error above: the value is looked up verbatim as a name.

Where the behaviour comes from

src/collection_manager.cpp — identical in v30.1, v30.2 and the v31 branch:

Option<bool> get_stopword_set(const std::map<std::string, std::string>& req_params,
                              std::string& stopwords_set) {
    const auto stopword_it = req_params.find("stopwords");
    if(stopword_it != req_params.end()) {
        stopwords_set = stopword_it->second;              // whole value, no split
        if(!StopwordsManager::get_instance().stopword_exists(stopwords_set)) {
            return Option<bool>(404, "Could not find the stopword set named `" + stopwords_set + "`.");
        }
    }
    return Option<bool>(true);
}

StopwordsManager::stopword_exists() is a plain stopword_configs.find(name), and Collection applies exactly one stopword_struct_t. There is no split(',') on this value anywhere in the repo, and test/stopwords_manager_test.cpp only ever passes a single set name.

The rest of the ecosystem says "one set"

  • typesense-api-spec/openapi.yml: stopwords: type: string"Name of the stopwords set to apply for this search"
  • The Stopwords guide this row links to: "we can pass a stopword set via the stopwords parameter", and its only example is "stopwords": "stopword_set1"
  • typesense-go (generated from the spec): Stopwords *string
  • typesense-js: stopwords?: string

Possible origin

  • #280 introduced the row as "A comma separated list of words to be dropped from the search query" — which did not match the parameter either (it takes a set name, not words).
  • #349 reworded wordsstopword sets but kept the "A comma separated list" opening from that earlier sentence.

The wording has also propagated downstream: typesense-python types the parameter as stopwords: Union[str, List[str]] with the pre-#349 docstring, so a list passes type checking but 404s at runtime.

Request

Either:

  1. Fix the docs — describe it as the name of a single stopword set (matching the OpenAPI spec and the Stopwords guide), or
  2. Implement multi-set support in the engine, if that was the intent — applying several sets in one search is genuinely useful (e.g. a shared base set plus a per-profile set) and today it can only be emulated by materialising a merged set.

Happy to send a docs PR for option 1 if that is the preferred route.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the versioned API search pages, such as 30.2/api/search.html, and compare their stopwords entry with the Stopwords guide and typesense-api-spec/openapi.yml. Update the documentation to describe one stopword set name, then check the affected 27.1–30.2 pages for consistent wording and verify that the examples match the API behavior.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.