nextcloud / nextcloud/fulltextsearch
fulltextsearch:index hangs, repeating "- RAM: 566.something"
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 234
- Forks
- 64
- Avg merge
- 6h 18m
- Merged PRs (30d)
- 10
Description
I'm trying to perform my first index, and I'm almost definitely stress testing this whole FullTextSearch stack while doing so — I've got a 500 GB dataset.
The data itself is "external storage," which is CIFS storage served from Windows Server 2012 R2, mounted via cifs entries in /etc/fstab as follows:
## Windows Shares
//<server>/Q-Data/shared /NetShares/v cifs credentials=/NetShares/.smbcredentials,iocharset=utf8,sec=ntlm,file_mode=0777,dir_mode=0777 0 0
//<server>/R-Data /NetShares/r cifs credentials=/NetShares/.smbcredentials,iocharset=utf8,sec=ntlm,file_mode=0777,dir_mode=0777 0 0
//<server>/J-Data /NetShares/j cifs credentials=/NetShares/.smbcredentials,iocharset=utf8,sec=ntlm,file_mode=0777,dir_mode=0777 0 0
//<server>/W-Data /NetShares/w cifs credentials=/NetShares/.smbcredentials,iocharset=utf8,sec=ntlm,file_mode=0777,dir_mode=0777 0 0
I've gotten it to index for one user successfully, but then as soon as it gets to the next user it gets stuck filling the RAM (?) once it gets to - RAM: 566.43410491943.
If I try to continue indexing by simply doing occ fulltextsearch:index again this happens:
root@c:/var/ncdata# occ fulltextsearch:index
indexing Files.
USER: blr@ieitransit.com
- RAM: 23.89949798584
- RAM: 30.564025878906
- RAM: 35.104431152344
- RAM: 37.639022827148
- RAM: 41.782417297363
- RAM: 46.028549194336
- RAM: 49.148544311523
- RAM: 50.502540588379
- RAM: 54.82398223877
- RAM: 59.080146789551
- RAM: 60.91951751709
- RAM: 65.728569030762
- RAM: 66.85343170166
- RAM: 70.643966674805
- RAM: 73.079170227051
- RAM: 77.160293579102
- RAM: 80.570121765137
- RAM: 83.654678344727
- RAM: 91.375785827637
- RAM: 96.923522949219
- RAM: 102.13726043701
- RAM: 110.47479248047
- RAM: 113.87605285645
- RAM: 116.39376831055
- RAM: 119.73052215576
- RAM: 123.79248809814
- RAM: 128.93239593506
- RAM: 132.1827545166
- RAM: 135.75133514404
- RAM: 141.4944152832
- RAM: 149.27917480469
- RAM: 159.47479248047
- RAM: 164.93907928467
- RAM: 169.28715515137
- RAM: 173.63157653809
- RAM: 177.92807769775
- RAM: 181.32400512695
- RAM: 184.85238647461
- RAM: 188.70569610596
- RAM: 192.77071380615
- RAM: 196.81725311279
- RAM: 204.74087524414
- RAM: 207.80700683594
- RAM: 211.59580993652
- RAM: 486.29839324951
- RAM: 508.19902038574
- RAM: 277.52561950684
- RAM: 272.46606445312
- RAM: 279.18186187744
- RAM: 287.55152893066
- RAM: 285.63525390625
- RAM: 291.30284118652
- RAM: 301.78762817383
- RAM: 306.75387573242
- RAM: 310.88928222656
- RAM: 316.43890380859
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
- RAM: 566.42482757568
and then it'll keep repeating that last number and going nowhere.
The only relevant entry in the log file is:
{
"reqId": "M8HADbAjjBympXqBfFxw",
"level": 1,
"time": "2018-05-08T11:53:44-04:00",
"remoteAddr": "",
"user": "--",
"app": "admin_audit",
"method": "--",
"url": "--",
"message": "Console command executed: fulltextsearch:index",
"userAgent": "--",
"version": "13.0.2.1"
}
I've reset the entire index more than once and started over, but it always ends up hanging, repeating the same - RAM: line over and over. I let it sit for over 24 hours doing so, just to be certain…
I imagine this number itself isn't particularly significant (I have no clue what it even refers to), but nevertheless I don't know where to start troubleshooting more.
I want to reference this issue I'm having, https://github.com/nextcloud/fulltextsearch_elasticsearch/issues/22 because I believe they may be related.
Lastly, while it does sound to me like this may be something to do with resource limits, I have definitely given the VM running Nextcloud / FullTextSearch / ElasticSearch, etc. enough resources: 32 GB RAM and 8 Xeon E5 CPU cores. If it's as simple as telling something it's allowed to use more RAM, so be it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the occ fulltextsearch:index entry point and reproduce the repeated - RAM: 566.42482757568 output against the described external CIFS storage setup. Compare the behavior with the referenced fulltextsearch_elasticsearch issue and inspect the command's RAM reporting; done means identifying why indexing stops progressing and documenting or correcting the hang.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, php
- Domain
- backend, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100