SharePoint Search API returns inaccurate and inconsistent results count across pages (TotalRows)

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

SharePoint REST API

Developer environment

No response

What browser(s) / client(s) have you tested

• 💥 Internet Explorer
• 💥 Microsoft Edge
• 💥 Google Chrome
• 💥 FireFox
• 💥 Safari
• mobile (iOS/iPadOS)
• mobile (Android)
• not applicable
• other (enter in the "Additional environment details" area below)

Additional environment details

• SharePoint Online
• Custom SPFx web part
• SPFx version: 1.23
• Search endpoint: /_api/search/postquery
• Results are displayed using paging with 10 results per page
• The issue occurs on multiple pages, multiple sites and on multiple tenants
• First observed: late august

Describe the bug / error

We are observing significantly inaccurate and inconsistent values for PrimaryQueryResult.RelevantResults.TotalRows when using the SharePoint Search REST API (/_api/search/postquery) and paging through the results using StartRow.

The same search query is executed for each page. The query parameters remain unchanged except for StartRow.
With StartRow = 0, returned TotalRows = 200
With StartRow = 10, returned TotalRows = 400
With StartRow = 20, returned TotalRows = 400
With StartRow = 30, returned TotalRows = 500
As we navigate further through the result set, the reported TotalRows progressively changes and eventually becomes much closer to the actual size of the result set (5,359 items)

The values are also observed to change when the same request is repeated over time.

We understand that TotalRows may be an estimate and therefore do not expect the value to always be exact. However, we would expect the estimate to remain reasonably stable when navigating through pages of the same query and provide a reasonable approximation of the result set size.

We also observe a similar issue when querying graph search (https://graph.microsoft.com/v1.0/search/query)
In that case the hitsContainers.total is wrong

Steps to reproduce

1. Execute a SharePoint Search REST API POST request against /_api/search/postquery
   Use a query matching a relatively large result set and request 10 results:
   RowLimit: 10
StartRow: 0
   Read: TotalRows

2. Execute exactly the same search query with:
   StartRow: 10
TotalRows changes

3. Repeat with:
   StartRow: 20
TotalRows changes

4. Continue paging through the same query. The reported TotalRows progressively changes and becomes much closer to the actual result-set size

The search criteria and other request parameters remain unchanged throughout these requests; only StartRow changes.

Expected behavior

We understand that TotalRows can be an estimated rather than exact value. However, for the same query we would expect TotalRows to provide a reasonably representative estimate of the result-set size and to remain reasonably stable when navigating between consecutive pages.

For a result set containing thousands of items, an initial estimate of only 200 items is too inaccurate to be useful for pagination.

Changing only StartRow should also not cause the estimated total to increase dramatically as progressively more pages are requested.

Ideally, TotalRows should either provide a reasonably stable approximation of the total result count across pages or
provide a documented mechanism that clients can use to obtain a sufficiently accurate count for implementing pagination.