manticoresoftware / manticoresoftware/manticoresearch

phrase_boundary_step is ignored with morphology

Open
#259 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
12k
Forks
642
Avg merge
3d 12h
Merged PRs (30d)
47

Description

Describe the environment

Manticore Search version: 3.0.0

OS version: 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64

Describe the problem

Description of the issue:

phrase_boundary_step is ignored with morphology

Steps to reproduce:

source src_xxx:src_pattern {
        sql_query = SELECT 1, 'hello . world' AS name
}

index xxx {
        type = plain
        source = src_xxx
        path = /db/sphinx/xxx/xxx

        phrase_boundary = .
        phrase_boundary_step = 1000
        index_exact_words = 1
        morphology = lemmatize_ru_all,lemmatize_en_all
}

With this config call select:

mysql> select * from xxx where match('"hello world"');
+------+
| id   |
+------+
|    1 |
+------+

But it should not return result.

Now remove this lines from config:

        index_exact_words = 1
        morphology = lemmatize_ru_all,lemmatize_en_all

Than rebuild index, and then call select:

mysql> select * from xxx where match('"hello world"');
Empty set (0.00 sec)

mysql> select * from xxx where match('"hello world"~2000');
+------+
| id   |
+------+
|    1 |
+------+

That works as expected.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the shown source and index configuration, then query the phrase with morphology enabled and disabled. Trace the phrase_boundary_step handling in the search and morphology code; done means "hello world" returns no result with the boundary step, while "hello world"~2000 still matches as shown.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.