Unable to export large account

Hello, I hope it is going well for you.

I will let you in on a little of my experience with using user migration tool. It sounds like a nice idea but so far it has been only frustration for me. The user I am trying to export has about 27BG of data. I know that is a lot but I was hoping it would work.

The first issue I had was with exporting. It would run a few minutes (probably less then 10 and then say it is done successfully). But nextcloud always said the file was zero KB and would not let me download. I worked through some errors in the log in the admin account but none of them ended up being related. I ran it a few more times while watching the log and watching the file size. Sometimes I deleted the file in between runs. I think now it was just not done when it said it was. Also, I think it was not really a zero byte file. It was just nextcloud that said it was.

I finally let one run for a day or so and the file ended up at about 18GB (I would hope I watched the file size for a while to make sure it was done, but I am not sure if I did). NextCloud still said it was zero byte but it would let me download it. I thought maybe that was good but I decided to try again since it seemed kind of small. On the second run after a while I had a 37GB file. I am not sure if I deleted the file in between runs. That seems kind of large but I moved it over and tried to import it.

I had the same kind of user interface experience. It runs for a few minutes (maybe 5-7) then says "Import completed successfully", but when I look around I do not find any files. In this case there is an item in the log. It says "ValueError: Invalid or uninitialized Zip object". That makes me think it is a bad file.

It is not hard to get more then 27BGs of data but it feels like the migration tool is not made to work with that large of a data set.

I saw someone talk about running it form the command line but was not able to find documentation on that. Could someone point me to that? I will try running it form the command line.

I think it would be nice to have examples of running it from the command line on the readme (https://github.com/nextcloud/user_migration#readme).

Also, it would be nice to have a more detailed report. Like the total number of files imported or exported and the total size. Or just something to help determine if it ran successfully.

I am using "Nextcloud Hub 3 (25.0.4)"

Thanks for your help.

Start with the repository README and the reported command-line migration request. Review the export and import behavior for large archives, including the reported zero-size display and “Invalid or uninitialized Zip object” log entry. Done should include a reliable outcome for large migrations and, if the requested scope includes it, command-line examples and useful file-count or size reporting.