Azure / Azure/azure-sdk-for-python
[Cosmos] Full Text Search query tests investigation
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 3.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 193
Description
Out live tests pipeline recently started running into some issues with some tests that were addressed in this PR: https://github.com/Azure/azure-sdk-for-python/pull/41595. Although most of these were able to be fixed, the two tests affected in the `test_query_hybrid_search` files (test_hybrid_search_weighted_reciprocal_rank_fusion, test_hybrid_search_queries) were instead "patched" to include the different behaviors they can have.
For one of the particular tests for instance, sometimes we receive 15 items, sometimes we receive 13. For others, sometimes the ranking is not what we expect, so we can't compare against a set ordered list of indices like previously.
The idea here would be to circle with the Query team folks and figure out exactly what is happening with these tests (since they have been around and unchanged for the most part since the initial preview of full text search). Ideally, we are able to explain this behavior, and either improve the current changes we have after the PR above or develop a fixed ordered output like we previously had. Regardless, finding the answer to this behavior should be the goal.
The PR above should also provide the differences from before and after to test against, in order to verify against the previous behavior and to verify the constant difference in outputs from the queries. Since FTS queries to full partition fanouts, an easy way to find out where each individual query is going is by using the `response_hook` kwarg on the query being executed, and taking a look at each of the partition ids - document count - activity ids.
Contributor guide
Assessment
This issue has not been assessed yet.