nextcloud / nextcloud/previewgenerator

Preview generator does not generate any preview after mass data migrated images

Open
#124 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
520
Forks
55
PR merge metrics
No merged PRs in 30d

Description

It seems there is a "bug" in Preview Generator app which prevents generation of previews after you performed a mass data migration (copy of images to data folder, perform occ files:scan --all).

I want to import very many image files into a new installation of Next Cloud and I would like to have the previews pre-generated. I do not want to upload several gigabytes via pc client or webdav (after that preview generation works, at least I tested webdav upload and previews for the uploaded files were generated). I think this is why there is this mass data migration available.

After mass data migration, the files are in NextCloud, you could access them via webdav or gui - all seems fine. But Preview Generator "does not see" them. It will not even generate previews for that 3 images which comes along default installation.

Below is the procedure I used for reproducing the issue. I used NextCloud as a snap installation.

  1. Fresh install of nextcloud, no previews exists so far:
# ls -al /var/snap/nextcloud/common/nextcloud/data/appdata_ocm286yevx6a/preview
total 8
drwxr-xr-x 2 root root 4096 Aug 22 14:23 .
drwxr-xr-x 7 root root 4096 Aug 22 14:24 ..
  1. Install Preview Generator app via GUI, no previews exists so far:
# ls -al /var/snap/nextcloud/common/nextcloud/data/appdata_ocm286yevx6a/preview
total 8
drwxr-xr-x 2 root root 4096 Aug 22 14:23 .
drwxr-xr-x 8 root root 4096 Aug 22 14:26 ..
  1. nextcloud.occ preview:pre-generate, no previews exists so far:
# ls -al /var/snap/nextcloud/common/nextcloud/data/appdata_ocm286yevx6a/preview
total 8
drwxr-xr-x 2 root root 4096 Aug 22 14:23 .
drwxr-xr-x 8 root root 4096 Aug 22 14:26 ..
  1. copy some jpg to data directory and perform files scan so the files are visible in nextcloud (2017-* were copied there, the rest 3 are coming with default installation)
# ls -al /var/snap/nextcloud/common/nextcloud/data/user/files/Photos/
total 28296
drwxr-xr-x 2 root root    4096 Aug 22 14:30 .
drwxr-xr-x 4 root root    4096 Aug 22 14:10 ..
-rw-r----- 1 root root 1576687 Aug 22 14:30 2017-01-01_123325.jpg
-rw-r----- 1 root root 1623375 Aug 22 14:30 2017-01-01_123328.jpg
-rw-r----- 1 root root 3681799 Aug 22 14:30 2017-01-01_123335.jpg
-rw-r----- 1 root root 3218276 Aug 22 14:30 2017-01-01_123725.jpg
-rw-r----- 1 root root 3628021 Aug 22 14:30 2017-01-01_123742.jpg
-rw-r----- 1 root root 3895594 Aug 22 14:30 2017-01-01_123853.jpg
-rw-r----- 1 root root 3029542 Aug 22 14:30 2017-01-01_175627.jpg
-rw-r----- 1 root root 3027073 Aug 22 14:30 2017-01-01_175717.jpg
-rw-r----- 1 root root 2900706 Aug 22 14:30 2017-01-01_175730.jpg
-rw-r--r-- 1 root root  819766 Aug 22 14:10 Coast.jpg
-rw-r--r-- 1 root root  585219 Aug 22 14:10 Hummingbird.jpg
-rw-r--r-- 1 root root  955026 Aug 22 14:10 Nut.jpg
# nextcloud.occ files:scan --all
Starting scan for user 1 out of 1 (user)

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5       | 16    | 00:00:00     |
+---------+-------+--------------+
  1. now we have 16 files in Next cloud - if I would go to GUI now I would see all the images there, but I will not do it and perform preview generation:
# nextcloud.occ preview:pre-generate
#
  1. No previews were generated:
# ls -al /var/snap/nextcloud/common/nextcloud/data/appdata_ocm286yevx6a/preview
total 8
drwxr-xr-x 2 root root 4096 Aug 22 14:23 .
drwxr-xr-x 8 root root 4096 Aug 22 14:26 ..
  1. Copied 2 files via webdav and tried initiate preview generation, all previews for that 2 images were generated:
# nextcloud.occ preview:pre-generate
# 
# ls -al /var/snap/nextcloud/common/nextcloud/data/appdata_ocm286yevx6a/preview
total 16
drwxr-xr-x 4 root root 4096 Aug 22 14:36 .
drwxr-xr-x 8 root root 4096 Aug 22 14:26 ..
drwxr-xr-x 2 root root 4096 Aug 22 14:36 124
drwxr-xr-x 2 root root 4096 Aug 22 14:36 125
# find . -type f
./125/128-96.jpg
./125/256-256-crop.jpg
./125/683-512.jpg
./125/32-32-crop.jpg
./125/1024-1024-crop.jpg
./125/2048-1536-max.jpg
./125/1536-1536-crop.jpg
./125/85-64.jpg
./125/43-32.jpg
./125/512-384.jpg
./125/64-48.jpg
./125/1365-1024.jpg
./125/32-24.jpg
./125/64-64-crop.jpg
./125/512-512-crop.jpg
./125/341-256.jpg
./125/1024-768.jpg
./125/256-192.jpg
./125/171-128.jpg
./125/128-128-crop.jpg
./124/128-96.jpg
./124/256-256-crop.jpg
./124/683-512.jpg
./124/32-32-crop.jpg
./124/1024-1024-crop.jpg
./124/2048-1536-max.jpg
./124/1536-1536-crop.jpg
./124/85-64.jpg
./124/43-32.jpg
./124/512-384.jpg
./124/64-48.jpg
./124/1365-1024.jpg
./124/32-24.jpg
./124/64-64-crop.jpg
./124/512-512-crop.jpg
./124/341-256.jpg
./124/1024-768.jpg
./124/256-192.jpg
./124/171-128.jpg
./124/128-128-crop.jpg

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the documented snap installation steps, including files:scan --all and preview:pre-generate, and compare the copied files with the two uploaded through WebDAV. Check the preview output under the documented appdata.../preview path; done means pre-generation creates previews for files added by mass migration as it does for WebDAV uploads.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.