nextcloud / nextcloud/nextcloudpi
nc-previews 3 out of 4 processes stop generating previews
Open
Nobody has claimed this yet.
bug
needs verification
- Dominant language
- Shell
- Stars
- 2.9k
- Forks
- 318
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
when executed on rock64, 3 out of 4 processes stop generating previews.
Terminal output
2019-12-29T18:37:01+00:00 Scanning folder /<username>/files/Media/Photos/2019
2019-12-29T18:37:13+00:00 Scanning folder /<username>/files/Media/Photos/2019/2019 .01_France
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 In Folder.php line 162:
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 [OCP\Files\NotPermittedException]
2019-12-29T18:40:05+00:00 Could not create folder
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 Exception trace:
2019-12-29T18:40:05+00:00 () at /var/www/nextcloud/lib/private/Files/Node/Folde r.php:162
2019-12-29T18:40:05+00:00 OC\Files\Node\Folder->newFolder() at /var/www/nextclo ud/lib/private/Files/AppData/AppData.php:155
2019-12-29T18:40:05+00:00 OC\Files\AppData\AppData->newFolder() at /var/www/nex tcloud/lib/private/Preview/Generator.php:419
2019-12-29T18:40:05+00:00 OC\Preview\Generator->getPreviewFolder() at /var/www/ nextcloud/lib/private/Preview/Generator.php:115
2019-12-29T18:40:05+00:00 OC\Preview\Generator->getPreview() at /var/www/nextcl oud/lib/private/PreviewManager.php:205
2019-12-29T18:40:05+00:00 OC\PreviewManager->getPreview() at /var/www/nextcloud /apps/previewgenerator/lib/Command/Generate.php:184
2019-12-29T18:40:05+00:00 OCA\PreviewGenerator\Command\Generate->parseFile() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:171
2019-12-29T18:40:05+00:00 OCA\PreviewGenerator\Command\Generate->parseFolder() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:169
2019-12-29T18:40:05+00:00 OCA\PreviewGenerator\Command\Generate->parseFolder() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:145
2019-12-29T18:40:05+00:00 OCA\PreviewGenerator\Command\Generate->generatePathPr eviews() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:11 5
2019-12-29T18:40:05+00:00 OCA\PreviewGenerator\Command\Generate->execute() at / var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:255
2019-12-29T18:40:05+00:00 Symfony\Component\Console\Command\Command->run() at / var/www/nextcloud/3rdparty/symfony/console/Application.php:901
2019-12-29T18:40:05+00:00 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:262
2019-12-29T18:40:05+00:00 Symfony\Component\Console\Application->doRun() at /va r/www/nextcloud/3rdparty/symfony/console/Application.php:145
2019-12-29T18:40:05+00:00 Symfony\Component\Console\Application->run() at /var/ www/nextcloud/lib/private/Console/Application.php:213
2019-12-29T18:40:05+00:00 OC\Console\Application->run() at /var/www/nextcloud/c onsole.php:97
2019-12-29T18:40:05+00:00 require_once() at /var/www/nextcloud/occ:11
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 preview:generate-all [-p|--path [PATH]] [--] [<user_id >]
2019-12-29T18:40:05+00:00
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 In Folder.php line 162:
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 [OCP\Files\NotPermittedException]
2019-12-29T18:40:34+00:00 Could not create folder
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 Exception trace:
2019-12-29T18:40:34+00:00 () at /var/www/nextcloud/lib/private/Files/Node/Folde r.php:162
2019-12-29T18:40:34+00:00 OC\Files\Node\Folder->newFolder() at /var/www/nextclo ud/lib/private/Files/AppData/AppData.php:155
2019-12-29T18:40:34+00:00 OC\Files\AppData\AppData->newFolder() at /var/www/nex tcloud/lib/private/Preview/Generator.php:419
2019-12-29T18:40:34+00:00 OC\Preview\Generator->getPreviewFolder() at /var/www/ nextcloud/lib/private/Preview/Generator.php:115
2019-12-29T18:40:34+00:00 OC\Preview\Generator->getPreview() at /var/www/nextcl oud/lib/private/PreviewManager.php:205
2019-12-29T18:40:34+00:00 OC\PreviewManager->getPreview() at /var/www/nextcloud /apps/previewgenerator/lib/Command/Generate.php:184
2019-12-29T18:40:34+00:00 OCA\PreviewGenerator\Command\Generate->parseFile() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:171
2019-12-29T18:40:34+00:00 OCA\PreviewGenerator\Command\Generate->parseFolder() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:169
2019-12-29T18:40:34+00:00 OCA\PreviewGenerator\Command\Generate->parseFolder() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:145
2019-12-29T18:40:34+00:00 OCA\PreviewGenerator\Command\Generate->generatePathPr eviews() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:11 5
2019-12-29T18:40:34+00:00 OCA\PreviewGenerator\Command\Generate->execute() at / var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:255
2019-12-29T18:40:34+00:00 Symfony\Component\Console\Command\Command->run() at / var/www/nextcloud/3rdparty/symfony/console/Application.php:901
2019-12-29T18:40:34+00:00 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:262
2019-12-29T18:40:34+00:00 Symfony\Component\Console\Application->doRun() at /va r/www/nextcloud/3rdparty/symfony/console/Application.php:145
2019-12-29T18:40:34+00:00 Symfony\Component\Console\Application->run() at /var/ www/nextcloud/lib/private/Console/Application.php:213
2019-12-29T18:40:34+00:00 OC\Console\Application->run() at /var/www/nextcloud/c onsole.php:97
2019-12-29T18:40:34+00:00 require_once() at /var/www/nextcloud/occ:11
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 preview:generate-all [-p|--path [PATH]] [--] [<user_id >]
...
the last 4th process continues to load cpu and apparently yield new previews:
visible in htop as /bin/bash /usr/local/bin/ncc preview:generate-all -n -v -p /<username>/files/Media/Photos/2019
btw, in ncp-report one should probably consider marking "trusted_domains" also as sensitive value
System information
NextCloudPi diagnostics
NextCloudPi version v1.20.1
distribution Debian Buster with Armbian Linux \l
automount yes
USB devices sda sdb
datadir /media/myCloudDrive/ncdata
data in SD no
data filesystem btrfs
data disk usage 38G/224G
rootfs usage 4.9G/29G
swapfile /var/swap
dbdir /media/myCloudDrive/ncdatabase
Nextcloud check ok
Nextcloud version 16.0.4.1
HTTPD service up
PHP service up
MariaDB service up
Redis service up
Postfix service up
internet check ok
port check 80 open
port check 443 open
IP ***REMOVED SENSITIVE VALUE***
gateway ***REMOVED SENSITIVE VALUE***
interface eth0
certificates ***REMOVED SENSITIVE VALUE***
NAT loopback no
uptime 10days
Nextcloud configuration
{
"system": {
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
},
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "16.0.4.1",
"overwrite.cli.url": "https:\/\/cloud.serkez.xyz\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"memcache.local": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 0,
"password": "***REMOVED SENSITIVE VALUE***"
},
"tempdirectory": "\/media\/myCloudDrive\/ncdata\/tmp",
"mail_smtpmode": "smtp",
"mail_smtpauthtype": "LOGIN",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"overwriteprotocol": "https",
"maintenance": false,
"loglevel": "2",
"log_type": "file",
"logfile": "\/media\/myCloudDrive\/ncdata\/nextcloud.log",
"jpeg_quality": "60",
"theme": "",
"data-fingerprint": "7c6b1dc19ed02a8b3f0cdae2ece076bc",
"htaccess.RewriteBase": "\/",
"preview_max_y": "2048",
"preview_max_x": "2048",
"mail_sendmailmode": "smtp",
"mail_smtpsecure": "ssl",
"mail_smtpauth": 1,
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***"
}
}
HTTPd logs
Database logs
Nextcloud logs
{"reqId":"XgfcSzejx852qgDc@yTbEwAAAA0","level":2,"time":"2019-12-28T22:50:54+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/apps\/spreed\/api\/v1\/signaling\/settings","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgfcUDejx852qgDc@yTbFAAAAA8","level":2,"time":"2019-12-28T22:51:04+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQFCqN63hPs7kDeAyN3wAAAAU","level":2,"time":"2019-12-29T11:37:58+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/apps\/spreed\/api\/v1\/signaling\/settings","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQFCqN63hPs7kDeAyN3gAAAAM","level":2,"time":"2019-12-29T11:37:58+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQGCqN63hPs7kDeAyN4AAAAAc","level":2,"time":"2019-12-29T11:38:02+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":3,"time":"2019-12-29T18:37:24+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"unlink(\/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343637\/455-256.jpg): No such file or directory at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#228","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:37:47+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:37:56+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":3,"time":"2019-12-29T18:38:24+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"unlink(\/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343649\/256-144.jpg): No such file or directory at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#228","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:38:26+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:08+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:12+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:22+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"8uHalDzuMptTpOBxU2yS","level":3,"time":"2019-12-29T18:39:31+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"filesize(): stat failed for \/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343659\/455-256.jpg at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#169","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:31+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:39:35+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":3,"time":"2019-12-29T18:39:53+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stat(): stat failed for \/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343664\/256-256-crop.jpg at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#143","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:39:53+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":2,"time":"2019-12-29T18:40:05+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:40:10+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the preview:generate-all command and the stack trace through apps/previewgenerator/lib/Command/Generate.php, lib/private/Preview/Generator.php, and lib/private/Files/Node/Folder.php. Reproduce the failure on the reported path and determine why three processes stop after the NotPermittedException. Done means preview generation completes reliably for the affected folder without the processes stopping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, shell
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100