nextcloud / nextcloud/fulltextsearch
Searching in trashbin over-loads server
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 234
- Forks
- 64
- Avg merge
- 6h 18m
- Merged PRs (30d)
- 10
Description
I have FullTextSearch, FullTextSearch_ElasticSearch, Files_FullTextSearch and Full text search - Files - Tesseract OCR installed on Nextcloud-24.04 on Ubuntu-20.04 with PHP-7.4.3 and MariaDB-10.5.16. The server is a quad-core with 16 GB RAM. Searching works fine unless done so from the trashbin ('deleted files'). In that situation, I observe the following:
- the search entry text box becomes unresponsive immediately after typing the very first character, a "blindly" typed second character does appear later on, though,.
- the number of php-fpm threads jumps from 22 to 125
- system load as provided by 'top' increases from 0.4 to values of up to 40
- memory consumption reaches 95% of 16 GB
- swap consumption does not seem to increase
- the website does not respond
After approximately 10 minutes, the search results are presented and the system returns to a normal load and memory consumption, and also the number of php-fpm threads drops back to the original 22.
Furthermore the search term can be changed by adding or deleting characters and the results adapt within approximately one minute, even -- and this seems remarkable -- if the search term is completely deleted and replaced. Server load land number of php-fpm threads increase only slightly during this period. The search entry text box remains unresponsive, nevertheless.
Neither PHP7.4-fpm nor Elasticsearch show anything noteworthy in their logs. In the web server error log there are repeated entries of type 2022/08/18 09:06:01 [error] 118#118: *622146 connect() to unix:/run/php/php7.4-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.1.102, server: lxcnextcloud, request: "GET /apps/files_trashbin/preview?fileId=682653&file=SofortUpload%2Fpix%2F_xport%2F20200907_7.jpg%2F20200907_7.jpg.d1599668497&c=1599668497 HTTP/1.0", upstream: "fastcgi://unix:/run/php/php7.4-fpm.sock:", host: "nextcloud.DOMAIN", referrer: "https://nextcloud.DOMAIN/apps/files/?dir=/&view=trashbin". To me, they look related to general system over-load.
The user data consists of 316GB under 'files/' and 190GB under 'files_trashbin/'. I remember having stumbled over this issue already on earlier versions of Nextcloud but have never taken a closer look at it up until now.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce a search from the deleted files view with the listed full-text-search components, using the reported Nextcloud, PHP, MariaDB, and Elasticsearch versions. Start by correlating PHP-FPM and web-server logs during the request; done means identifying the cause of the excessive workers and load and confirming that trashbin searches return without overwhelming the server.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, mariadb, php
- Domain
- backend, performance, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100