S3 external storage created via `occ files_external:create` does not detect remote file changes
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
Hi,
Issue:
Nextcloud is supposed to follow any change in S3 directory on access, as configured:
However, when I add/delete a file on S3 side, Nextcloud does not update when I access the directory containing the file. Only if I do a manual scan occ files:scan or if I try to access the file in Nextcloud, which results in nothing, then after a browser refresh the list is correct.
Workaround:
I added a file scan every 5 min in cron, but this may not be good for S3 server to be interrogated for nothing, and we have a 5min delay.
Steps to reproduce
- deploy Nextcloud
- deploy external storage app
- configure S3 remote
- add file in S3
- see if the file appear in Nextcloud immediately
=> the file is not present
Expected behavior
Any access on Nextcloud will show correct info of file in remote.
Nextcloud Server version
30
Operating system
Other
PHP engine version
PHP 8.2
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"htaccess.RewriteBase": "\/",
"memcache.local": "\\OC\\Memcache\\APCu",
"apps_paths": [
{
"path": "\/var\/www\/html\/apps",
"url": "\/apps",
"writable": false
},
{
"path": "\/var\/www\/html\/custom_apps",
"url": "\/custom_apps",
"writable": true
}
],
"debug": true,
"check_data_directory_permissions": false,
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"forwarded_for_headers": [
"HTTP_X_FORWARDED_FOR"
],
"skeletondirectory": "\/dev\/null",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"password": "***REMOVED SENSITIVE VALUE***",
"port": 6379
},
"upgrade.disable-web": true,
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": "***REMOVED SENSITIVE VALUE***"
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "30.0.6.2",
"overwrite.cli.url": "http:\/\/localhost",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
"filelocking.enabled": false
}
}
List of activated Apps
Enabled:
- activity: 3.0.0
- bruteforcesettings: 3.0.0
- cloud_federation_api: 1.13.0
- comments: 1.20.1
- dav: 1.31.1
- federatedfilesharing: 1.20.0
- files: 2.2.0
- files_downloadlimit: 3.0.0
- files_external: 1.22.0
- files_pdfviewer: 3.0.0
- files_sharing: 1.22.0
- logreader: 3.0.0
- lookup_server_connector: 1.18.0
- notifications: 3.0.0
- oauth2: 1.18.1
- password_policy: 2.0.0
- provisioning_api: 1.20.0
- serverinfo: 2.0.0
- settings: 1.13.0
- support: 2.0.0
- survey_client: 2.0.0
- systemtags: 1.20.0
- text: 4.1.0
- theming: 2.6.0
- twofactor_backupcodes: 1.19.0
- updatenotification: 1.20.0
- user_ldap: 1.21.0
- viewer: 3.0.0
- workflowengine: 2.12.0
Enabled:
- activity: 3.0.0
- bruteforcesettings: 3.0.0
- cloud_federation_api: 1.13.0
- comments: 1.20.1
- dav: 1.31.1
- federatedfilesharing: 1.20.0
- files: 2.2.0
- files_downloadlimit: 3.0.0
- files_external: 1.22.0
- files_pdfviewer: 3.0.0
- files_sharing: 1.22.0
- logreader: 3.0.0
- lookup_server_connector: 1.18.0
- notifications: 3.0.0
- oauth2: 1.18.1
- password_policy: 2.0.0
- provisioning_api: 1.20.0
- serverinfo: 2.0.0
- settings: 1.13.0
- support: 2.0.0
- survey_client: 2.0.0
- systemtags: 1.20.0
- text: 4.1.0
- theming: 2.6.0
- twofactor_backupcodes: 1.19.0
- updatenotification: 1.20.0
- user_ldap: 1.21.0
- viewer: 3.0.0
- workflowengine: 2.12.0Disabled:
- admin_audit: 1.20.0
- app_api: 4.0.6 (installed 4.0.6)
- circles: 30.0.0 (installed 30.0.0)
- contactsinteraction: 1.11.0 (installed 1.11.0)
- dashboard: 7.10.0 (installed 7.10.0)
- encryption: 2.18.0
- federation: 1.20.0 (installed 1.20.0)
- files_reminders: 1.3.0 (installed 1.3.0)
- files_trashbin: 1.20.1 (installed 1.20.1)
- files_versions: 1.23.0 (installed 1.23.0)
- firstrunwizard: 3.0.0 (installed 3.0.0)
- nextcloud_announcements: 2.0.0 (installed 2.0.0)
- photos: 3.0.2 (installed 3.0.2)
- privacy: 2.0.0 (installed 2.0.0)
- recommendations: 3.0.0 (installed 3.0.0)
- related_resources: 1.5.0 (installed 1.5.0)
- sharebymail: 1.20.0 (installed 1.20.0)
- suspicious_login: 8.0.0
- twofactor_nextcloud_notification: 4.0.0
- twofactor_totp: 12.0.0-dev
- user_status: 1.10.0 (installed 1.10.0)
- weather_status: 1.10.0 (installed 1.10.0)
- webhook_listeners: 1.1.0-dev (installed 1.1.0-dev)
Nextcloud Signing status
No errors have been found.
Nextcloud Logs
Additional info
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the files_external app flow used by occ files_external:create when an S3 directory is accessed, and compare it with the behavior after occ files:scan. Reproduce the external file add/delete sequence from the issue and inspect the related storage listing and cache paths. Done means directory access reflects remote S3 changes without requiring a manual scan or opening the missing file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100