nextcloud / nextcloud/fulltextsearch

Search results limited by size

Open
#606 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
234
Forks
64
Avg merge
6h 18m
Merged PRs (30d)
10

Description

Steps to reproduce

The results should show 10 results per page but for some reason the count is limited by the size parameter .
For example a query say there are 180 results . This should lead to 18 pages of 10 results . But this is not true in every case . Some pages return for example only 6 results , especially for results where additional information shown (i.e, the part of the text where the match is ). This is reproducible with a query via the api and the size parameter is set to 10 or so.

{"providers\":\"all\",\"search\":\"K6401\",\"page\":1,\"options\":\{\"files_local\":\"\",\"files_external\":\"\",\"files_group_folders\":\"\",\"files_extension\":\"\"\},\"size\":10\}

-> "meta" : { "count" : 6, "maxScore" : 25, "time" : 1064, "timedOut" : false, "total" : 235 },

{\"providers\":\"all\",\"search\":\"K6401\",\"page\":10,\"options\":\{\"files_local\":\"\",\"files_external\":\"\",\"files_group_folders\":\"\",\"files_extension\":\"\"\},\"size\":10\}

->"meta" : { "count" : 10, "maxScore" : 25, "time" : 450, "timedOut" : false, "total" : 235 },

Server configuration detail

Operating system: Linux 5.10.12-1-default #1 SMP Sat Jan 30 19:15:49 UTC 2021 (a3c8888) x86_64

Webserver: Apache (fpm-fcgi)

Database: mysql 10.5.8

PHP version:

7.4.15
Modules loaded: Core, date, libxml, pcre, filter, hash, Reflection, SPL, session, SimpleXML, standard, xml, cgi-fcgi, mysqlnd, apcu, bcmath, bz2, ctype, curl, dom, enchant, mbstring, fileinfo, ftp, gd, gettext, gmp, ice, iconv, imagick, intl, json, lzf, exif, memcached, mysqli, openssl, pcntl, PDO, pdo_mysql, pdo_sqlite, zlib, posix, readline, redis, smbclient, sockets, sqlite3, tidy, tokenizer, uuid, xmlreader, xmlwriter, zip, Phar, libsmbclient, Zend OPcache, xdebug

Nextcloud version: 20.0.7 - 20.0.7.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: unknown

Signing status

Array
(
)

List of activated apps 
Enabled:
 - activity: 2.13.4
 - admin_audit: 1.10.0
 - announcementcenter: 4.0.1
 - apporder: 0.11.0
 - bookmarks: 4.0.8
 - bruteforcesettings: 2.0.1
 - calendar: 2.1.3
 - checksum: 1.0.1
 - circles: 0.20.6
 - cloud_federation_api: 1.3.0
 - comments: 1.10.0
 - contacts: 3.4.3
 - contactsinteraction: 1.1.0
 - dashboard: 7.0.0
 - dav: 1.16.2
 - epubreader: 1.4.5
 - event_update_notification: 1.2.0
 - extract: 1.3.0
 - federatedfilesharing: 1.10.2
 - federation: 1.10.1
 - files: 1.15.0
 - files_external: 1.11.1
 - files_fulltextsearch: 20.0.0
 - files_fulltextsearch_tesseract: 20.0.1
 - files_linkeditor: 1.1.3
 - files_markdown: 2.3.1
 - files_pdfviewer: 2.0.1
 - files_rightclick: 0.17.0
 - files_sharing: 1.12.2
 - files_trackdownloads: 1.9.0
 - files_trashbin: 1.10.1
 - files_versions: 1.13.0
 - files_videoplayer: 1.9.0
 - firstrunwizard: 2.9.0
 - flowupload: 1.1.2
 - fulltextsearch: 20.0.0
 - fulltextsearch_elasticsearch: 20.0.0
 - groupfolders: 8.2.0
 - issuetemplate: 0.7.0
 - logreader: 2.5.0
 - lookup_server_connector: 1.8.0
 - maps: 0.1.8
 - music: 1.0.0
 - news: 15.3.1
 - nextcloud_announcements: 1.9.0
 - notes: 4.0.2
 - notifications: 2.8.0
 - oauth2: 1.8.0
 - passman: 2.3.6
 - password_policy: 1.10.1
 - phonetrack: 0.6.6
 - photos: 1.2.3
 - polls: 1.7.5
 - previewgenerator: 3.1.1
 - pride: 0.0.3
 - privacy: 1.4.0
 - provisioning_api: 1.10.0
 - serverinfo: 1.10.0
 - settings: 1.2.0
 - sharebymail: 1.10.0
 - side_menu: 1.22.1
 - social: 0.4.2
 - spreed: 10.0.5
 - support: 1.3.0
 - survey_client: 1.8.0
 - suspicious_login: 3.2.1
 - systemtags: 1.10.0
 - talk_matterbridge: 1.22.0
 - talk_simple_poll: 1.2.0
 - tasks: 0.13.6
 - telephoneprovider: 1.0.3
 - text: 3.1.0
 - theming: 1.11.0
 - twofactor_backupcodes: 1.9.0
 - updatenotification: 1.10.0
 - user_status: 1.0.1
 - viewer: 1.4.0
 - weather_status: 1.0.0
 - workflowengine: 2.2.0
Disabled:
 - accessibility
 - deck
 - duplicatefinder
 - encryption
 - metadata
 - nextbackup
 - recommendations
 - user_ldap
Configuration (config/config.php) 
{
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "nextcloud.mybase.selfhost.de",
        "cloudnext.mybase.selfhost.de"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "tempdirectory": "\/var\/tmp\/nextcloudtemp",
    "overwrite.cli.url": "https:\/\/nextcloud.mybase.selfhost.de",
    "knowledgebaseenabled": true,
    "dbtype": "mysql",
    "version": "20.0.7.1",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "updater.release.channel": "beta",
    "appstore.experimental.enabled": true,
    "maintenance": false,
    "loglevel": 3,
    "debug": false,
    "htaccess.RewriteBase": "\/",
    "activity_use_cached_mountpoints": true,
    "session_keepalive": true,
    "session_lifetime": 86400,
    "versions_retention_obligation": "auto",
    "trashbin_retention_obligation": "auto",
    "enable_previews": true,
    "preview_max_x": 2560,
    "preview_max_y": 1600,
    "preview_max_filesize_image": 200,
    "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
    "preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir ",
    "enabledPreviewProviders": [
        "OC\\Preview\\PNG",
        "OC\\Preview\\JPEG",
        "OC\\Preview\\GIF",
        "OC\\Preview\\BMP",
        "OC\\Preview\\XBitmap",
        "OC\\Preview\\MP3",
        "OC\\Preview\\TXT",
        "OC\\Preview\\MarkDown",
        "OC\\Preview\\TIFF",
        "OC\\Preview\\SVG",
        "OC\\Preview\\Postscript",
        "OC\\Preview\\OpenDocument",
        "OC\\Preview\\PDF",
        "OC\\Preview\\Font",
        "OC\\Preview\\Epub",
        "OC\\Preview\\StarOffice",
        "OC\\Preview\\HEIC",
        "OC\\Preview\\KRITA"
    ],
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpmode": "smtp",
    "mail_smtpauthtype": "LOGIN",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "log_type": "syslog",
    "logfile": "\/var\/log\/nextcloud.log",
    "logfilemode": 416,
    "log_rotate_size": 104857600,
    "syslog_tag": "Nextcloud",
    "memcache.distributed": "\\OC\\Memcache\\Redis",
    "memcache.local": "\\OC\\Memcache\\Redis",
    "filelocking.enabled": "true",
    "filelocking.ttl": 3600,
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "redis": {
        "host": "***REMOVED SENSITIVE VALUE***",
        "port": 6379,
        "timeout": 0,
        "dbindex": 0,
        "password": "***REMOVED SENSITIVE VALUE***"
    },
    "memcached_servers": [
        [
            "127.0.0.1",
            11211
        ]
    ],
    "data-fingerprint": "790e9a77b6ee76b2af214c8f58d75dfd",
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpauth": 1,
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpsecure": "tls",
    "mysql.utf8mb4": true,
    "allow_user_to_change_display_name": true,
    "auth.bruteforce.protection.enabled": true,
    "ldapIgnoreNamingRules": false,
    "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
    "theme": "",
    "app_install_overwrite": [
        "fulltextsearch",
        "fulltextsearch_elasticsearch",
        "files_fulltextsearch",
        "files_fulltextsearch_tesseract",
        "passman",
        "pride",
        "checksum",
        "files_markdown",
        "telephoneprovider",
        "circles",
        "camerarawpreviews",
        "files_external_gdrive",
        "music",
        "facerecognition"
    ]
}

Are you using external storage, if yes which one: local/smb/sftp/...

Are you using encryption:

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

Client configuration

Browser: Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Firefox/85.0

Operating system:

Logs

Web server error log 
Insert your web server log here
Nextcloud log 
Insert your Nextcloud log here
Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

fulltextsearch results page 1
fulltextsearch results page 10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two API requests in the issue with the reported search term and size of 10, comparing the returned count and total across pages. Trace the full-text search API and Elasticsearch-backed search behavior involved; done means each page returns up to 10 results and pagination reflects the full result total.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, php
Domain
api, backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.