[Bug]: NC 30 Web UI does not show all copied files / folders after copying large amount of files and folders but appear to be there via WebDAV
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
Copying a large amount of files and folders using the web interface leads to a web view presentation where it seems as not all of the files and folders where copied. Only a fraction of the ae presented.
Steps to reproduce
Steps I used to reproduce this issue (at least in my installation). All steps are done via the web interface.
- Mount external NC storage using External Storage feature in AIO NC 30
- Navigate to folder containing large amounts of files and folders.
- Select all and use Copy/Move to copy all to a folder residing on the local NC. The amount is about 35000 files in approximitely 30 folders and 20 GB in size.
- Waiting about 24 hours to let the copying process finishing the job.
- After the copying process only a fraction of files and folders are shown.
- A WebDAV connection to the local AIO NC 30 shows all files and folder and a random check proves them all as accessible. See screen shot below.
The bug exists for a longer time period and I noticed it the first time after I started again using NC in version 24.
Expected behavior
All files and folder are indexed and presented in the web interface.
Nextcloud Server version
30
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.2
Web server
Apache (supported)
Database engine version
PostgreSQL
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"one-click-instance": true,
"one-click-instance.user-limit": 100,
"memcache.local": "\\OC\\Memcache\\APCu",
"apps_paths": [
{
"path": "\/var\/www\/html\/apps",
"url": "\/apps",
"writable": false
},
{
"path": "\/var\/www\/html\/custom_apps",
"url": "\/custom_apps",
"writable": true
}
],
"appsallowlist": false,
"check_data_directory_permissions": false,
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"password": "***REMOVED SENSITIVE VALUE***",
"port": 6379
},
"overwritehost": "nextcloud.your.domain.tld",
"overwriteprotocol": "https",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"localhost",
"nextcloud.your.domain.tld"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "pgsql",
"version": "29.0.7.1",
"overwrite.cli.url": "https:\/\/nextcloud.your.domain.tld\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"maintenance": false,
"updatedirectory": "\/nc-updater",
"loglevel": "2",
"log_type": "file",
"logfile": "\/var\/www\/html\/data\/nextcloud.log",
"log_rotate_size": "10485760",
"log.condition": {
"apps": [
"admin_audit"
]
},
"preview_max_x": "2048",
"preview_max_y": "2048",
"jpeg_quality": "60",
"enabledPreviewProviders": {
"1": "OC\\Preview\\Image",
"2": "OC\\Preview\\MarkDown",
"3": "OC\\Preview\\MP3",
"4": "OC\\Preview\\TXT",
"5": "OC\\Preview\\OpenDocument",
"6": "OC\\Preview\\Movie",
"7": "OC\\Preview\\Krita",
"0": "OC\\Preview\\Imaginary"
},
"enable_previews": true,
"upgrade.disable-web": true,
"mail_smtpmode": "smtp",
"trashbin_retention_obligation": "auto, 30",
"versions_retention_obligation": "auto, 30",
"activity_expire_days": "30",
"simpleSignUpLink.shown": false,
"share_folder": "\/Shared",
"one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
"upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
"allow_local_remote_servers": true,
"davstorage.request_timeout": 3600,
"htaccess.RewriteBase": "\/",
"dbpersistent": false,
"files_external_allow_create_new_local": false,
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
"default_phone_region": "DE",
"mail_smtpsecure": "ssl",
"mail_sendmailmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpauth": 1,
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"maintenance_window_start": 100,
"preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",
"auth.bruteforce.protection.enabled": true,
"ratelimit.protection.enabled": true
}
}
List of activated Apps
Enabled:
- activity: 2.21.1
- admin_audit: 1.19.0
- analytics: 5.0.1
- analytics_sourcepack: 0.0.3
- app_api: 3.2.1
- appointments: 2.1.12
- assistant: 1.1.0
- calendar: 4.7.16
- circles: 29.0.0-dev
- cloud_federation_api: 1.12.0
- collectives: 2.14.4
- comments: 1.19.0
- contacts: 6.0.0
- contactsinteraction: 1.10.0
- context_chat: 2.2.1
- dashboard: 7.9.0
- dav: 1.30.1
- deck: 1.13.2
- drawio: 3.0.3
- federatedfilesharing: 1.19.0
- federation: 1.19.0
- files: 2.1.1
- files_antivirus: 5.5.7
- files_automatedtagging: 1.19.0
- files_bpm: 1.5.0
- files_downloadlimit: 2.0.0
- files_external: 1.21.0
- files_fulltextsearch: 29.0.1
- files_pdfviewer: 2.10.0
- files_reminders: 1.2.0
- files_sharing: 1.21.0
- files_trashbin: 1.19.0
- files_versions: 1.22.0
- firstrunwizard: 2.18.0
- forms: 4.3.1
- fulltextsearch: 29.0.1
- fulltextsearch_elasticsearch: 29.0.1
- integration_collaboard: 1.0.10
- integration_openstreetmap: 1.0.12
- logreader: 2.14.0
- lookup_server_connector: 1.17.0
- mail: 3.7.8
- maps: 1.4.0
- multiboards: 1.0.4
- nextcloud-aio: 0.6.0
- nextcloud_announcements: 1.18.0
- notes: 4.11.0
- notifications: 2.17.0
- notify_push: 0.7.0
- oauth2: 1.17.1
- password_policy: 1.19.0
- photos: 2.5.0
- privacy: 1.13.0
- provisioning_api: 1.19.0
- recommendations: 2.1.0
- related_resources: 1.4.0
- richdocuments: 8.4.7
- serverinfo: 1.19.0
- settings: 1.12.0
- sharebymail: 1.19.0
- spreed: 19.0.9
- support: 1.12.0
- survey_client: 1.17.0
- systemtags: 1.19.0
- tables: 0.8.0
- tasks: 0.16.1
- text: 3.10.1
- theming: 2.4.0
- twofactor_backupcodes: 1.18.0
- twofactor_totp: 11.0.0-dev
- user_status: 1.9.0
- viewer: 2.3.0
- weather_status: 1.9.0
- whiteboard: 1.0.3
- workflowengine: 2.11.0
- xwiki: 0.1.2
Disabled:
- bruteforcesettings: 2.9.0
- encryption: 2.17.0
- suspicious_login: 7.0.0
- user_ldap: 1.20.0
Nextcloud Signing status
No errors have been found.
Nextcloud Logs
This log is flooded with this two different messages:
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"L5bKTvW1Q8toW4YjVu0Q","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"POST","url":"/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
{"reqId":"zclf2gNgxs2WIdHwDHnk","level":2,"time":"2024-10-11T12:08:29+00:00","remoteAddr":"172.31.0.1","user":"admin","app":"no app in context","method":"GET","url":"/apps/dashboard/","message":"AppManger::checkAppForUser - can't decode group IDs: Failed to construct console command 'OCA\\LdapWriteSupport\\Command\\GroupAdminsToLdap': Could not resolve OCA\\User_LDAP\\Helper! Class \"OCA\\User_LDAP\\Helper\" does not exist\nyes - json error code: 4","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0","version":"29.0.7.1","data":[]}
Additional info
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the web interface's Copy/Move flow and compare its resulting listing with the WebDAV listing after the reported 35,000-file transfer. Reproduce the issue on Nextcloud 30 and trace the file listing and indexing behavior; done means every copied file and folder is shown in the web interface as well as through WebDAV.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, php, postgresql
- Domain
- backend, frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100