typesense / typesense/typesense

group_by pagination stops returning results beyond some limit in Typesense

Open
#2,622 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

pre-triage
Dominant language
C++
Stars
26.6k
Forks
973
Avg merge
18h 45m
Merged PRs (30d)
4

Description

Bug Description

When using group_by with a high number of unique groups, Typesense only returns results up to the max_query_groups limit. Requests for higher page numbers return empty grouped_hits, even though there are more unique groups available in the collection.

Reproduction Steps

curl -X GET "http://TS_URL/collections/products/documents/search"
-H "X-TYPESENSE-API-KEY:API_KEY"
-G --data-urlencode 'q=*'
--data-urlencode 'query_by=product_id'
--data-urlencode 'filter_by=(brand_name:="Nike")'
--data-urlencode 'per_page=10'
--data-urlencode 'page=12'
--data-urlencode 'group_by=product_id'
--data-urlencode 'group_limit=1'
--data-urlencode 'sort_by=product_id:asc'

Expected vs Actual

It should give result for page 12, give 10 records

Actual behavior
{"facet_counts":[],"found":13061,"found_docs":51947,"grouped_hits":[],"out_of":435977,"page":12,"request_params":{"collection_name":"products","first_q":"","per_page":10,"q":""},"search_cutoff":false,"search_time_ms":96}

It gives grouped_hits as blank, for page 11 it works.

Environment

  • Typesense version: e.g. v29.0

Schema / Configuration

Include any relevant schema definitions, JSON payloads, or config files:

schema = {
            "name": "products",
            "enable_nested_fields": True,
            "fields": [
                {"name": "id", "type": "string", "facet": True, "sort": True},
                {"name": "product_id", "type": "string", "sort": True},
       {
                    "name": "images",
                    "type": "object[]",
                    "fields": [
                        {"name": "product_image_id", "type": "string", "sort": True},
                        {"name": "is_primary", "type": "bool", "sort": True},
                        {"name": "large_url", "type": "string"}
                    ]
                },

                {
                    "name": "store_details",
                    "type": "object",
                    "fields": [
                        {"name": "store_id", "type": "string", "sort": True},
                        
                    ]
                }
            ],
            "default_sorting_field": "price_amount",
            "num_shards": 4,
        }

Additional Context

For other product brands some works till 150 page for 200 page does not work. Total pages in this case was 1800+

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the request against the search endpoint with group_by=product_id, group_limit=1, per_page=10, and page values around 11 and 12. Trace the grouped pagination and max_query_groups handling, then verify that page 12 returns grouped_hits rather than an empty array while preserving the reported filtering and sorting behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.