[Bug]: Rotating AWS keys in external storage config causes loss of shares

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

28.0.12

Operating System: Linux 6.8.0-1032-aws x86_64

CPU: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz (2 threads)

Memory: 7.66 GB

When rotating AWS access keys for external S3 storage configuration, we needed to change the key and secret for the existing external storage in the config. Changing this caused all of the configured shares to be empty.

Checking the database, the share table is the same, the only table that seemed to change was the oc_external_config that reflects the key change, the oc_storages, which added two new records for the two external storages that had their key changed. The files tables may have also been changed.

We reviewed a number of existing bug reports and community requests that referenced issues with losing connection to the external storage and the shares disappearing. This seemed relevant, but was more related to an amount of time that external storage was disconnected.

To try to get our shares back, we performed a restore of a backup from AWS of the EC2 instance, but since the restore would just fail when it tried to connect with the old keys, we stopped apache/php/mysql/cron immediately after startup. We then checked that the oc_storages table hadn't been modified yet, and modified the oc_external_config table manually to insert the new keys and secrets. This way, presumably, nextcloud would startup with the correct keys and would not "detect" a problem.

Even that didn't work. As soon as we started apache/cron/php and logged in, it added two new records for external storage again, and all of the shares were empty.

So it appears that any change in this manner to the external storage configuration invalidates all shares. We must be able to rotate aws api keys. It also seems excessive and non-intuitive that making a configuration change to an existing external storage (not a remove or add), would cause such chaos.

Currently, we are stuck recreating hundreds of shares. If there is a better way to relink these, I would be overjoyed to know it so that work can continue.

Thanks!

Steps to reproduce

1. Make a change to the external storage AWS api keys configuration
2. All shares are empty

Expected behavior

1. Make a change to the external storage AWS api keys configuration

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "xxx.xxxxx.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.12.2",
        "overwrite.cli.url": "https:\/\/xxx.xxxxx.com",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": "true",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0
        },
        "enable_previews": false,
        "session_lifetime": 86400,
        "skeletondirectory": "",
        "tempdirectory": "\/var\/www\/nextcloud\/data\/tmp",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpsecure": "ssl",
        "theme": "",
        "loglevel": 0,
        "app_install_overwrite": [
            "files_external_dropbox"
        ],
        "trashbin_retention_obligation": "auto, 14",
        "oupdater.release.channel": "stable",
        "onlyoffice": {
            "verify_peer_off": true
        },
        "updater.release.channel": "stable",
        "default_phone_region": "CA",
        "maintenance_window_start": 9
    }
}

List of activated Apps

Enabled:
  - activity: 2.20.0
  - bruteforcesettings: 2.8.0
  - calendar: 4.7.16
  - camerarawpreviews: 0.8.6
  - circles: 28.0.0
  - cloud_federation_api: 1.11.0
  - comments: 1.18.0
  - contactsinteraction: 1.9.0
  - dashboard: 7.8.0
  - dav: 1.29.2
  - external: 5.3.1
  - federatedfilesharing: 1.18.0
  - files: 2.0.0
  - files_external: 1.20.0
  - files_pdfviewer: 2.9.0
  - files_reminders: 1.1.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - firstrunwizard: 2.17.0
  - forms: 4.3.4
  - groupfolders: 16.0.12
  - integration_google: 3.1.1
  - integration_onedrive: 3.3.1
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - nextcloud_announcements: 1.17.0
  - notifications: 2.16.0
  - oauth2: 1.16.4
  - onlyoffice: 9.5.0
  - password_policy: 1.18.0
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - quicknotes: 0.8.23
  - recommendations: 2.0.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - spreed: 18.0.13
  - survey_client: 1.16.0
  - systemtags: 1.18.0
  - text: 3.9.2
  - theming: 2.3.0
  - twofactor_backupcodes: 1.17.0
  - updatenotification: 1.18.0
  - user_status: 1.8.1
  - viewer: 2.2.0
  - weather_status: 1.8.0
  - workflowengine: 2.10.0
Disabled:
  - admin_audit: 1.18.0
  - contacts: 5.5.3 (installed 5.5.3)
  - drawio: 3.0.3 (installed 3.0.3)
  - duplicatefinder: 1.6.1 (installed 1.6.1)
  - encryption: 2.16.0
  - federation: 1.18.0 (installed 1.11.0)
  - files_rightclick: 0.15.1 (installed 1.6.0)
  - geoblocker: 0.5.15 (installed 0.5.15)
  - integration_dropbox: 3.0.3 (installed 3.0.3)
  - integration_reddit: 2.0.5 (installed 2.0.5)
  - integration_whiteboard: 0.0.14 (installed 0.0.14)
  - mail: 3.7.17 (installed 3.7.17)
  - photos: 2.4.0 (installed 1.3.0)
  - ransomware_protection: 1.14.0 (installed 1.14.0)
  - support: 1.11.1 (installed 1.4.0)
  - suspicious_login: 6.0.0
  - twofactor_totp: 10.0.0-beta.2 (installed 6.4.1)
  - unsplash: 3.0.3 (installed 3.0.3)
  - user_ldap: 1.19.0
  - workflow_media_converter: 1.11.7 (installed 1.11.7)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Additional info

No response

Start by reproducing AWS external storage key rotation on Nextcloud 28/29 and compare the oc_external_config, oc_storages, and share tables before and after the change. Confirm that updating credentials preserves the existing external storage records and configured shares, then verify the behavior with Apache, PHP, and cron running.