nextcloud / nextcloud/server

[Bug]: NC 30 Web UI does not show all copied files / folders after copying large amount of files and folders but appear to be there via WebDAV

Open
#48,664 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 30-feedback bug feature: files
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

Copying a large amount of files and folders using the web interface leads to a web view presentation where it seems as not all of the files and folders where copied. Only a fraction of the ae presented.

Steps to reproduce

Steps I used to reproduce this issue (at least in my installation). All steps are done via the web interface.

  1. Mount external NC storage using External Storage feature in AIO NC 30
  2. Navigate to folder containing large amounts of files and folders.
  3. Select all and use Copy/Move to copy all to a folder residing on the local NC. The amount is about 35000 files in approximitely 30 folders and 20 GB in size.
  4. Waiting about 24 hours to let the copying process finishing the job.
  5. After the copying process only a fraction of files and folders are shown.
  6. A WebDAV connection to the local AIO NC 30 shows all files and folder and a random check proves them all as accessible. See screen shot below.

The bug exists for a longer time period and I noticed it the first time after I started again using NC in version 24.

Expected behavior

All files and folder are indexed and presented in the web interface.

Nextcloud Server version

30

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report
{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "appsallowlist": false,
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "nextcloud.your.domain.tld",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "nextcloud.your.domain.tld"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "29.0.7.1",
        "overwrite.cli.url": "https:\/\/nextcloud.your.domain.tld\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updatedirectory": "\/nc-updater",
        "loglevel": "2",
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": "10485760",
        "log.condition": {
            "apps": [
                "admin_audit"
            ]
        },
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita",
            "0": "OC\\Preview\\Imaginary"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": "30",
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "allow_local_remote_servers": true,
        "davstorage.request_timeout": 3600,
        "htaccess.RewriteBase": "\/",
        "dbpersistent": false,
        "files_external_allow_create_new_local": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "DE",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance_window_start": 100,
        "preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true
    }
}
List of activated Apps
Enabled:
  - activity: 2.21.1
  - admin_audit: 1.19.0
  - analytics: 5.0.1
  - analytics_sourcepack: 0.0.3
  - app_api: 3.2.1
  - appointments: 2.1.12
  - assistant: 1.1.0
  - calendar: 4.7.16
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - collectives: 2.14.4
  - comments: 1.19.0
  - contacts: 6.0.0
  - contactsinteraction: 1.10.0
  - context_chat: 2.2.1
  - dashboard: 7.9.0
  - dav: 1.30.1
  - deck: 1.13.2
  - drawio: 3.0.3
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.1
  - files_antivirus: 5.5.7
  - files_automatedtagging: 1.19.0
  - files_bpm: 1.5.0
  - files_downloadlimit: 2.0.0
  - files_external: 1.21.0
  - files_fulltextsearch: 29.0.1
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - firstrunwizard: 2.18.0
  - forms: 4.3.1
  - fulltextsearch: 29.0.1
  - fulltextsearch_elasticsearch: 29.0.1
  - integration_collaboard: 1.0.10
  - integration_openstreetmap: 1.0.12
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - mail: 3.7.8
  - maps: 1.4.0
  - multiboards: 1.0.4
  - nextcloud-aio: 0.6.0
  - nextcloud_announcements: 1.18.0
  - notes: 4.11.0
  - notifications: 2.17.0
  - notify_push: 0.7.0
  - oauth2: 1.17.1
  - password_policy: 1.19.0
  - photos: 2.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recommendations: 2.1.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.7
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - spreed: 19.0.9
  - support: 1.12.0
  - survey_client: 1.17.0
  - systemtags: 1.19.0
  - tables: 0.8.0
  - tasks: 0.16.1
  - text: 3.10.1
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - twofactor_totp: 11.0.0-dev
  - user_status: 1.9.0
  - viewer: 2.3.0
  - weather_status: 1.9.0
  - whiteboard: 1.0.3
  - workflowengine: 2.11.0
  - xwiki: 0.1.2
Disabled:
  - bruteforcesettings: 2.9.0
  - encryption: 2.17.0
  - suspicious_login: 7.0.0
  - user_ldap: 1.20.0
Nextcloud Signing status

No errors have been found.

Nextcloud Logs

This log is flooded with this two different messages:

{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
Additional info

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with the web interface's Copy/Move flow and compare its resulting listing with the WebDAV listing after the reported 35,000-file transfer. Reproduce the issue on Nextcloud 30 and trace the file listing and indexing behavior; done means every copied file and folder is shown in the web interface as well as through WebDAV.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, php, postgresql
Domain
backend, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.