nextcloud / nextcloud/fulltextsearch

Indexer crash at federated share

Open
#482 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
234
Forks
64
Avg merge
6h 18m
Merged PRs (30d)
10

Description

NC 14.0.8, FTS 1.1.1, FTS-ES 1.0.3, FTS-Files 1.1.3

while occ fulltextsearch:index the job aborts with the following traceback:

An unhandled exception has been thrown:
TypeError: Argument 1 passed to OC\OCS\DiscoveryService::discover() must be of the type string, null given, called in /srv/www/htdocs/nextcloud/apps/files_sharing/lib/External/Manager.php on line 382 and defined in /srv/www/htdocs/nextcloud/lib/private/OCS/DiscoveryService.php:65
Stack trace:
#0 /srv/www/htdocs/nextcloud/apps/files_sharing/lib/External/Manager.php(382): OC\OCS\DiscoveryService->discover(NULL, 'FEDERATED_SHARI...')
#1 /srv/www/htdocs/nextcloud/apps/files_sharing/lib/External/Manager.php(527): OCA\Files_Sharing\External\Manager->sendFeedbackToRemote(NULL, NULL, NULL, 'decline')
#2 /srv/www/htdocs/nextcloud/apps/files_sharing/lib/External/Storage.php(217): OCA\Files_Sharing\External\Manager->removeShare('e0-e376-1035-94...')
#3 /srv/www/htdocs/nextcloud/apps/files_sharing/lib/External/Storage.php(181): OCA\Files_Sharing\External\Storage->checkStorageAvailability()
#4 /srv/www/htdocs/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(378): OCA\Files_Sharing\External\Storage->hasUpdated('', 1521626402)
#5 /srv/www/htdocs/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php(398): OC\Files\Storage\Wrapper\Wrapper->hasUpdated('', 1521626402)
#6 /srv/www/htdocs/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(378): OC\Files\Storage\Wrapper\Availability->hasUpdated('', 1521626402)
#7 /srv/www/htdocs/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(378): OC\Files\Storage\Wrapper\Wrapper->hasUpdated('', 1521626402)
#8 /srv/www/htdocs/nextcloud/lib/private/Files/Cache/Watcher.php(126): OC\Files\Storage\Wrapper\Wrapper->hasUpdated('', 1521626402)
#9 /srv/www/htdocs/nextcloud/lib/private/Files/View.php(1332): OC\Files\Cache\Watcher->needsUpdate('', Object(OC\Files\Cache\CacheEntry))
#10 /srv/www/htdocs/nextcloud/lib/private/Files/View.php(1374): OC\Files\View->getCacheEntry(Object(OCA\Files_Trashbin\Storage), '', '/02051fe0-e376-...')
#11 /srv/www/htdocs/nextcloud/lib/private/Files/Node/Root.php(198): OC\Files\View->getFileInfo('/02051fe0-e376-...')
#12 /srv/www/htdocs/nextcloud/lib/private/Files/Node/Folder.php(134): OC\Files\Node\Root->get('/02051fe0-e376-...')
#13 /srv/www/htdocs/nextcloud/apps/files_fulltextsearch/lib/Service/FilesService.php(487): OC\Files\Node\Folder->get('SFB Dissipation')
#14 /srv/www/htdocs/nextcloud/apps/files_fulltextsearch/lib/Service/FilesService.php(392): OCA\Files_FullTextSearch\Service\FilesService->updateFilesDocument(Object(OCA\Files_FullTextSearch\Model\FilesDocument))
#15 /srv/www/htdocs/nextcloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(285): OCA\Files_FullTextSearch\Service\FilesService->generateDocument(Object(OCA\Files_FullTextSearch\Model\FilesDocument))
#16 /srv/www/htdocs/nextcloud/apps/fulltextsearch/lib/Service/IndexService.php(296): OCA\Files_FullTextSearch\Provider\FilesProvider->fillIndexDocument(Object(OCA\Files_FullTextSearch\Model\FilesDocument))
#17 /srv/www/htdocs/nextcloud/apps/fulltextsearch/lib/Service/IndexService.php(176): OCA\FullTextSearch\Service\IndexService->indexDocuments(Object(OCA\FullTextSearch_ElasticSearch\Platform\ElasticSearchPlatform), Object(OCA\Files_FullTextSearch\Provider\FilesProvider), Array, Object(OCA\FullTextSearch\Model\IndexOptions))
#18 /srv/www/htdocs/nextcloud/apps/fulltextsearch/lib/Command/Index.php(400): OCA\FullTextSearch\Service\IndexService->indexProviderContentFromUser(Object(OCA\FullTextSearch_ElasticSearch\Platform\ElasticSearchPlatform), Object(OCA\Files_FullTextSearch\Provider\FilesProvider), '02051fe0-e376-1...', Object(OCA\FullTextSearch\Model\IndexOptions))
#19 /srv/www/htdocs/nextcloud/apps/fulltextsearch/lib/Command/Index.php(264): OCA\FullTextSearch\Command\Index->indexProvider(Object(OCA\Files_FullTextSearch\Provider\FilesProvider), Object(OCA\FullTextSearch\Model\IndexOptions))
#20 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Command/Command.php(251): OCA\FullTextSearch\Command\Index->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /srv/www/htdocs/nextcloud/core/Command/Base.php(162): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(946): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(248): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FullTextSearch\Command\Index), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 /srv/www/htdocs/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /srv/www/htdocs/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#26 /srv/www/htdocs/nextcloud/console.php(96): OC\Console\Application->run()
#27 /srv/www/htdocs/nextcloud/occ(11): require_once('/srv/www/htdocs...')
#28 {main}stopping all running indexes

Checkbox 'index external files' is unchecked. Index doesn't get past this error, remains same when I rerun index. (is this setting relevant and really should skip federated shares?)

What can I do to skip inaccessible files, just to finish indexing?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with apps/files_sharing/lib/External/Manager.php around lines 382 and 527, then follow apps/files_sharing/lib/External/Storage.php and apps/files_fulltextsearch/lib/Service/FilesService.php. Reproduce with occ fulltextsearch:index and trace the null federated-share data; done means inaccessible federated files no longer abort the indexing run and the remaining files can be indexed.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, php
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.