nextcloud / nextcloud/groupfolders

Impossible to move an instance on S3/SWIFT while using GroupFolders

Open
#1,288 4 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

1. to develop bug integration low s3
Dominant language
PHP
Stars
343
Forks
106
Avg merge
2d 3h
Merged PRs (30d)
34

Description

We have an instance using S3 as primary storage. We we're planning to move it from one server to another, problem is that after moving it (different server, different providers for both servers and S3), any GroupFolders folder is...EMPTY.

Problem is inexistant if there is no GroupFolders involved, using the migration plan above.

Step of migration

While in maintenance mode:

  1. rsync all /public folder contents from one server to another
  2. mysqldump --single-transaction nextcloud > nextcloud.sql and import on the new server
  3. Adjust config.php to use new credentials
  4. Using rclone or mc (tried with both), sync all content from one S3 provider to another. At the end, the two S3 buckets match perfectly (number of objects, size, and each object matches it's MD5).
  5. Switch OFF maintenance mode on new server, and log in.

So at the end:

  • same files (code base)
  • same contents on S3 buckets
  • same SQL base (same MySQL version on old and new servers)
  • all operations done while NC was in maintenance mode

Power-up new instance by switching DNS entries (hosts file in my case), every group folder is EMPTY on the new instance. All folders are there, but empty. Everything that is NOT on a GroupFolder, is completely fine (files, shares, activities, users, calendars...). So the problem is somewhere with GroupFolder not detecting its contents on the new S3 bucket...

Also, every GroupFolder has a modified date as "Few seconds ago" just after log in to the new instance, which of course is not true. But maybe this is a lead to find the root cause.

We've tried this procedure moving from S3 > S3 or from S3 > SWIFT, the result is the same. We've switched many Nextcloud instances not using GroupFolders, using the above procedure, no problems.

Folder(s) scan doesn't help either:

php occ groupfolders:scan 1
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 1       | 0     | 00:00:00     |
+---------+-------+--------------+

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the migration with the reported S3 or SWIFT setup, checking config.php and the GroupFolders entry point php occ groupfolders:scan 1. Compare the scan results and modified dates before and after migration, then trace why GroupFolders sees the folders but no files. Done means migrated GroupFolders contents are detected and the scan reports the expected files.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, php
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.