manticoresoftware / manticoresoftware/manticoresearch

In the SPH_RANKER_HIT structure in the query_pos field, positions of service words are taken into account.

Open
#441 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
12k
Forks
642
Avg merge
3d 12h
Merged PRs (30d)
47

Description

In the SPH_RANKER_HIT structure in the query_pos field, positions of service words are taken into account.

For example, there is a query:
SELECT id, WEIGHT() FROM myindex WHERE MATCH(’“22К 734”~10|“4 1 208”~10’) OPTION ranker=my_ranker(’’);

When calling the plugin function XXX_update (), in the query_pos field for words 22K 734, there will be positions 1 and 2, respectively. And for the word '4' the position will be 4. This happened due to the fact that position 3 was assigned to the word '10', which is a service word and is the operand of the proximity operator.

The same effect can be observed if you call CALL KEYWORDS.
For example, CALL KEYWORDS (‘22К 734”~10|“4 1 208”~10’, ‘myindex’);
±-----±----------±-----------+
| qpos | tokenized | normalized |
±-----±----------±-----------+
| 1 | 22к | 22к |
| 2 | 734 | 734 |
| 3 | 10 | 10 |
| 4 | 4 | 4 |
| 5 | 1 | 1 |
| 6 | 208 | 208 |
| 7 | 10 | 10 |
±-----±----------±-----------+

This issue was discussed(question 3):
https://forum.manticoresearch.com/t/topic/637/2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported queries with ranker=my_ranker and CALL KEYWORDS, then trace how query_pos is populated for SPH_RANKER_HIT. Compare positions for service words such as the proximity operand 10 with ordinary terms. Done means service words no longer consume positions in query_pos or CALL KEYWORDS output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
backend, databases, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.