Error while backing up to external app data via SFTP

Expected Behavior

Get a completed full backup over SFTP connection to external storage.

Current Behavior

Backup fails after backing up random number of chunks.

Failure Information (for bugs)

Steps to Reproduce
Install Nextcloud Hub II 23.0.3.
Install external storage and configure it with SFTP connection using username and password.
Install backup app 1.0.6 and set backups and temporary data to go to external storage.
Try to create a full backup.

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

Nextcloud Hub II version: 23.0.3
Backup app version: 1.0.6
Nextcloud is installed on Ubuntu 20.04 LTS

I have configured app data to be on external storage configured with SFTP.
I can access the external storage with files.
Every time i run "sudo -u www-data php /var/www/nextcloud/occ backup:point:create" it will backup and create chunks on external storage.
I have a lot of data that is backed up so it takes several hours for a full backup.
But after a couple of hours it fails with message:
In ChunkService.php line 597:

Chunk data-8fe30df3-c083-4b2a-90ce-6878c0de32fe.zip not valid

Backup Output:

> maintenance mode: on
> initialization of the AppData
> initialization of the RestoringPoint: 20220414063407-full-lqJPD4Vv8mFlnvX based on NC23.0.3.2
> initialization of the storage
> preparation of the data to be stored in the restoring point
> preparation of internal data
> creating chunks
  * data: /opt/nextcloud/data/, 594386 files
    - data-bd4380e3-436a-4b08-9097-2187e099eb96
    - data-823febf1-db0a-4a67-a5fd-247e7be5fc54
    - data-460de213-0071-4874-a77b-9a4c3f584237
    - data-e7d87fbb-88bb-4bad-b7d1-c598a1d8615b
    - data-faae0aa3-5c23-43c5-8128-156c3047c8ac
    - data-23569b5b-ace1-4522-b496-5ce46816631f
    - data-8a5341ab-01f0-4021-9cf1-d4ae3841c171
    - data-c95887ca-27cb-458f-8243-bd5e2474c165
    - data-fa54cbbf-325e-4ed6-8fdc-7e70b9c6a474
    - data-3a67508b-4d3e-4e15-8374-bce11af48844
    - data-6a0fc0e4-4007-417c-95ca-f03217f5ff6e
    - data-a96fae05-59ca-4759-b6b7-245327a01b8e
    - data-715d3674-7144-4f26-aaa6-a75787776c48
    - data-f1ad0597-b6d7-46b8-a372-723acfb1ff7f
    - data-81993cdb-6a3a-4783-a66e-cf1b57b080ad
    - data-81016c23-c785-4213-82e7-093cca5fd4ab
    - data-cc1ac2b4-bef9-41aa-8561-6f54e6952d52
    - data-de2b3ff0-4303-4971-a015-0d3ebc2127ae
    - data-b5f905a4-be8d-48a2-994f-21bd0f8cdb29
    - data-1be3ce8c-cc2f-4a92-9c97-a8ea161dc8cc
    - data-4a53941c-0174-4ef7-a146-656eddc0f06a
    - data-489323de-33ba-4048-83ee-d1d89e2db5dd
    - data-a5268462-0469-416c-8d7a-45690f06beab
    - data-4ebc0d76-8b27-4569-920b-526214c5a7ad
    - data-148a2301-1bb8-4b0f-ae96-3ceedbe20e3e
    - data-f4c351dc-00ad-49d4-8cd5-f748103ea166
    - data-b4023dca-e092-411d-a000-faa150600c0b
    - data-8fe30df3-c083-4b2a-90ce-6878c0de32fe
> maintenance mode: off

In ChunkService.php line 597:

  Chunk data-8fe30df3-c083-4b2a-90ce-6878c0de32fe.zip not valid


backup:point:create [--output [OUTPUT]] [--generate-log] [--differential] [--] [<comment>]

Nextcloud log:

{"reqId":"di1OV9CSK6E4BqmFeNxD","level":3,"time":"2022-04-14T15:48:40+02:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fopen(sftpwrite://backups//nextcloudappdata/20220414063407-full-lqJPD4Vv8mFlnvX/data/data-8fe30df3-c083-4b2a-90ce-6878c0de32fe/.backup.data-8fe30df3-c083-4b2a-90ce-6878c0de32fe): failed to open stream: \"OCA\\Files_External\\Lib\\Storage\\SFTPWriteStream::stream_open\" call failed at /var/www/nextcloud/apps/files_external/lib/Lib/Storage/SFTP.php#387","userAgent":"--","version":"23.0.3.2","exception":{"Exception":"Error","Message":"fopen(sftpwrite://backups//nextcloudappdata/20220414063407-full-lqJPD4Vv8mFlnvX/data/data-8fe30df3-c083-4b2a-90ce-6878c0de32fe/.backup.data-8fe30df3-c083-4b2a-90ce-6878c0de32fe): failed to open stream: \"OCA\\Files_External\\Lib\\Storage\\SFTPWriteStream::stream_open\" call failed at /var/www/nextcloud/apps/files_external/lib/Lib/Storage/SFTP.php#387","Code":0,"Trace":[{"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/nextcloud/apps/files_external/lib/Lib/Storage/SFTP.php","line":387,"function":"fopen"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Common.php","line":206,"function":"fopen","class":"OCA\\Files_External\\Lib\\Storage\\SFTP","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Common","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/PermissionsMask.php","line":112,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\PermissionsMask","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php","line":274,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1169,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":706,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/Folder.php","line":198,"function":"file_put_contents","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFolder.php","line":89,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":551,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":476,"function":"finalizeChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":378,"function":"generateChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":148,"function":"fillChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/PointService.php","line":258,"function":"createChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Command/PointCreate.php","line":120,"function":"create","class":"OCA\\Backup\\Service\\PointService","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Command/Command.php","line":255,"function":"execute","class":"OCA\\Backup\\Command\\PointCreate","type":"->"},{"file":"/var/www/nextcloud/core/Command/Base.php","line":168,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":1009,"function":"run","class":"OC\\Core\\Command\\Base","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":273,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":149,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/lib/private/Console/Application.php","line":211,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/console.php","line":99,"function":"run","class":"OC\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/occ","line":11,"args":["/var/www/nextcloud/console.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/Log/ErrorHandler.php","Line":92,"CustomMessage":"--"}}
{"reqId":"di1OV9CSK6E4BqmFeNxD","level":3,"time":"2022-04-14T15:48:40+02:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fwrite() expects parameter 1 to be resource, bool given at /var/www/nextcloud/lib/private/Files/Storage/Common.php#208","userAgent":"--","version":"23.0.3.2","exception":{"Exception":"Error","Message":"fwrite() expects parameter 1 to be resource, bool given at /var/www/nextcloud/lib/private/Files/Storage/Common.php#208","Code":0,"Trace":[{"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Common.php","line":208,"function":"fwrite"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Common","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/PermissionsMask.php","line":112,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\PermissionsMask","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php","line":274,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1169,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":706,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/Folder.php","line":198,"function":"file_put_contents","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFolder.php","line":89,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":551,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":476,"function":"finalizeChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":378,"function":"generateChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":148,"function":"fillChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/PointService.php","line":258,"function":"createChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Command/PointCreate.php","line":120,"function":"create","class":"OCA\\Backup\\Service\\PointService","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Command/Command.php","line":255,"function":"execute","class":"OCA\\Backup\\Command\\PointCreate","type":"->"},{"file":"/var/www/nextcloud/core/Command/Base.php","line":168,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":1009,"function":"run","class":"OC\\Core\\Command\\Base","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":273,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":149,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/lib/private/Console/Application.php","line":211,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/console.php","line":99,"function":"run","class":"OC\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/occ","line":11,"args":["/var/www/nextcloud/console.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/Log/ErrorHandler.php","Line":92,"CustomMessage":"--"}}
{"reqId":"di1OV9CSK6E4BqmFeNxD","level":3,"time":"2022-04-14T15:48:40+02:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"fclose() expects parameter 1 to be resource, bool given at /var/www/nextcloud/lib/private/Files/Storage/Common.php#209","userAgent":"--","version":"23.0.3.2","exception":{"Exception":"Error","Message":"fclose() expects parameter 1 to be resource, bool given at /var/www/nextcloud/lib/private/Files/Storage/Common.php#209","Code":0,"Trace":[{"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Common.php","line":209,"function":"fclose"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Common","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/PermissionsMask.php","line":112,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\PermissionsMask","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Availability.php","line":274,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php","line":258,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1169,"function":"file_put_contents","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":706,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/Folder.php","line":198,"function":"file_put_contents","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFolder.php","line":89,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":551,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":476,"function":"finalizeChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":378,"function":"generateChunk","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/ChunkService.php","line":148,"function":"fillChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Service/PointService.php","line":258,"function":"createChunks","class":"OCA\\Backup\\Service\\ChunkService","type":"->"},{"file":"/var/www/nextcloud/apps/backup/lib/Command/PointCreate.php","line":120,"function":"create","class":"OCA\\Backup\\Service\\PointService","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Command/Command.php","line":255,"function":"execute","class":"OCA\\Backup\\Command\\PointCreate","type":"->"},{"file":"/var/www/nextcloud/core/Command/Base.php","line":168,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":1009,"function":"run","class":"OC\\Core\\Command\\Base","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":273,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/3rdparty/symfony/console/Application.php","line":149,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/lib/private/Console/Application.php","line":211,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/console.php","line":99,"function":"run","class":"OC\\Console\\Application","type":"->"},{"file":"/var/www/nextcloud/occ","line":11,"args":["/var/www/nextcloud/console.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/Log/ErrorHandler.php","Line":92,"CustomMessage":"--"}}

Start with apps/backup/lib/Service/ChunkService.php around lines 378, 476, 551, and 597, then trace the failed SFTP write at apps/files_external/lib/Lib/Storage/SFTP.php#387 and the fwrite handling in lib/private/Files/Storage/Common.php#208. Reproduce with occ backup:point:create using SFTP-backed external storage and determine why a chunk becomes invalid. Done means a full backup completes successfully without the reported invalid-chunk failure.