nextcloud / nextcloud/fulltextsearch_elasticsearch

Nextcloud 22 Elasticsearch 8.0 Compatability

Open
#178 5 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
89
Forks
44
Avg merge
1h 34m
Merged PRs (30d)
4

Description

We recently set up a brand new Elasticsearch 8.0 stack for other indexing needs in our org and saw that nextcloud had an app for full text search.
Great! I thought.
So I set it up and had some initial issues, but manually creating the index and attachment pipeline cleared it up such that the occ test and indexing ran without any errors!

So I ran my first search for some text i knew existed in some documents and..... Nothing

Did some debugging and eventually discovered that altering the filter 'term' queries to use owner.keyword, groups.keyword, and users.keyword seems to correct the behavior.

Looking at previous doc revisions, it seems elasticsearch may have changed the default behavior of text fields?
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html

here's the code I changed:

./fulltextsearch_elasticsearch/lib/Service/SearchMappingService.php
        private function generateSearchQueryAccess(IDocumentAccess $access): array {

                $query = [];
                $query[] = ['term' => ['owner.keyword' => $access->getViewerId()]];
                $query[] = ['term' => ['users.keyword' => $access->getViewerId()]];
                $query[] = ['term' => ['users.keyword' => '__all']];

                foreach ($access->getGroups() as $group) {
                        $query[] = ['term' => ['groups.keyword' => $group]];
                }

                foreach ($access->getCircles() as $circle) {
                        $query[] = ['term' => ['circles.keyword' => $circle]];
                }

                return $query;
        }

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in fulltextsearch_elasticsearch/lib/Service/SearchMappingService.php, especially generateSearchQueryAccess(), and review how its term filters behave with Elasticsearch 8.0. Use the reported occ test, indexing flow, and a search for known document text to verify the access filters; done means matching documents are returned without breaking access restrictions.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, php
Domain
backend, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.