Sync and deletion ends in runtime error

## Describe the bug
Users of ours encountered some errors while synchronizing files. This specific user is using the client to sync files. It started with a "404 Not Found" on "HEAD https...". One coincidence is, that all the fails were `.zip` files.

After we took a look we've found a lot of errors. The first error was a `ERR error doing PATCH request to data service error` with a `500` status code. This error came up while syncing the files.

```
2024/10/14 11:08:10 http: proxy error: context canceled
2024-10-14T11:08:10Z ERR error doing PATCH request to data service error="Patch \"http://localhost:9158/data/tus/7552f2ff-fa60-4106-ac07-8730501c6b66\": context canceled" pkg=rhttp request-id=f97965ab-337d-416a-be28-a1729477c86e service=frontend traceid=1e85ce1259f5b9e548424d9053a4a5a6
2024-10-14T11:08:10Z ERR http end="14/Oct/2024:11:08:10 +0000" host=127.0.0.1 method=PATCH pkg=rhttp proto=HTTP/1.1 service=storage-users size=15 start="14/Oct/2024:11:08:07 +0000" status=500 time_ns=3061060470 traceid=4b9ffc2dfc69f523b88fbba77bb1b87e uri=/data/tus/7552f2ff-fa60-4106-ac07-8730501c6b66 url=/7552f2ff-fa60-4106-ac07-8730501c6b66
2024-10-14T11:08:10Z ERR http end="14/Oct/2024:11:08:10 +0000" host=127.0.0.1 method=PATCH pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="14/Oct/2024:11:08:07 +0000" status=500 time_ns=3061408166 traceid=1e85ce1259f5b9e548424d9053a4a5a6 uri=/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwNDIzLCJpYXQiOjE3Mjg5MDQwMjMsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83NTUyZjJmZi1mYTYwLTQxMDYtYWMwNy04NzMwNTAxYzZiNjYifQ.4qmKtzvuaXUN5Z4-1YVb3jHIcZPedQFxCziUTc3Wis0 url=/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwNDIzLCJpYXQiOjE3Mjg5MDQwMjMsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy83NTUyZjJmZi1mYTYwLTQxMDYtYWMwNy04NzMwNTAxYzZiNjYifQ.4qmKtzvuaXUN5Z4-1YVb3jHIcZPedQFxCziUTc3Wis0
2024/10/14 11:08:10 http: proxy error: context canceled
2024-10-14T11:08:10Z ERR error doing PATCH request to data service error="Patch \"http://localhost:9158/data/tus/1b553492-cc4b-4b2c-bea1-d4b02eb795df\": context canceled" pkg=rhttp request-id=f27d065b-7439-4b5d-8d50-d90c2f88ec87 service=frontend traceid=af14d064b2b9012d4c3ce4dce555a45c
2024-10-14T11:08:10Z ERR http end="14/Oct/2024:11:08:10 +0000" host=127.0.0.1 method=PATCH pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="14/Oct/2024:11:08:05 +0000" status=500 time_ns=5771382483 traceid=af14d064b2b9012d4c3ce4dce555a45c uri=/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwMDkzLCJpYXQiOjE3Mjg5MDM2OTMsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy8xYjU1MzQ5Mi1jYzRiLTRiMmMtYmVhMS1kNGIwMmViNzk1ZGYifQ.QKm8V-l7j9cpUMaU-doxidbgRMLnUl8t3PhOv2adEyM url=/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwMDkzLCJpYXQiOjE3Mjg5MDM2OTMsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy8xYjU1MzQ5Mi1jYzRiLTRiMmMtYmVhMS1kNGIwMmViNzk1ZGYifQ.QKm8V-l7j9cpUMaU-doxidbgRMLnUl8t3PhOv2adEyM
2024/10/14 11:08:11 http: proxy error: context canceled
2024-10-14T11:08:11Z ERR error doing PATCH request to data service error="Patch \"http://localhost:9158/data/tus/c89ed49a-9f75-46d5-ae80-b7e8835e40fc\": context canceled" pkg=rhttp request-id=2eb641eb-80f4-4e11-8817-690a6b5d16e2 service=frontend traceid=f1eabf37d480d7aaaf38b698c9cf6300
2024-10-14T11:08:11Z ERR http end="14/Oct/2024:11:08:11 +0000" host=127.0.0.1 method=PATCH pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="14/Oct/2024:11:08:05 +0000" status=500 time_ns=5662717637 traceid=f1eabf37d480d7aaaf38b698c9cf6300 uri=/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwNDY5LCJpYXQiOjE3Mjg5MDQwNjksInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jODllZDQ5YS05Zjc1LTQ2ZDUtYWU4MC1iN2U4ODM1ZTQwZmMifQ.WDOoQMfCp2sItzikFypWdHcnPGD2zneI_aTn7HQmjUg url=/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwNDY5LCJpYXQiOjE3Mjg5MDQwNjksInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy9jODllZDQ5YS05Zjc1LTQ2ZDUtYWU4MC1iN2U4ODM1ZTQwZmMifQ.WDOoQMfCp2sItzikFypWdHcnPGD2zneI_aTn7HQmjUg
2024-10-14T11:08:15Z WRN http end="14/Oct/2024:11:08:15 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:08:15 +0000" status=404 time_ns=146176 traceid=1a897147aaa902e3eac297057c6ce669 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json
2024-10-14T11:08:15Z WRN http end="14/Oct/2024:11:08:15 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:1
```

Additional messages in connection with the sync were the following:

```
2024-10-14T11:08:44Z ERR failed to authenticate the request error="token is expired" authenticator=oidc path=/ocs/v2.php/cloud/activity service=proxy
2024-10-14T11:08:44Z ERR failed to authenticate the request error="token is expired" authenticator=oidc path=/ocs/v2.php/apps/notifications/api/v1/notifications service=proxy
```

```
2024-10-14T11:09:59Z ERR invalid transfer token error="datagateway: error validating transfer token: error parsing token: token is expired by 69h53m2s" pkg=rhttp request-id=061982d2-85c5-4cbe-933b-f24ded9cbe25 service=frontend token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4NjUyNjE3LCJpYXQiOjE3Mjg1NjYyMTcsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy8wY2YyZWRlMy1hOWUyLTQ3MzktOTNmOS03N2YzMTMxZmNhODAifQ.Nf6p9tOvkS2Hi87R9wf5wU5X-qxMsZLXBRSj7EwxcgQ traceid=1866c1dc58c6f92908ddb25c08ab6e27
```

We are using OIDC for authentication and S3 as storage backend. The users did try to logout and login to renew any buggy sessions.

After this failed try, he tried to reinitialize the synchronization and delete all files to start the sync completely new. We received the following errors:
```
2024-10-14T11:21:22Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:22Z ERR error listing grantees error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node="marshaling error: json: unsupported value: encountered a cycle via *node.Node" pkg=rgrpc service=storage-users traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:22Z ERR error reading permissions error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:23Z WRN http end="14/Oct/2024:11:21:23 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:23 +0000" status=404 time_ns=149812 traceid=2af2644db78c3f1d2a391c9606e3d8ba uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json
2024-10-14T11:21:23Z WRN http end="14/Oct/2024:11:21:23 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:23 +0000" status=404 time_ns=140204 traceid=a1dabfaa8652d7caf70a786f0032d116 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json url=/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing grantees error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node="marshaling error: json: unsupported value: encountered a cycle via *node.Node" pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error reading permissions error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f2/fa/43/d6/-c890-4962-b436-31c1095c0570: no such file or directory" node=f2fa43d6-c890-4962-b436-31c1095c0570 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/df/e2/f8/37/-572b-4d32-b3e3-18722a97546c: no such file or directory" node=dfe2f837-572b-4d32-b3e3-18722a97546c pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/08/a6/95/d4/-fc79-4ab1-b7da-4e69478a40dd: no such file or directory" node=08a695d4-fc79-4ab1-b7da-4e69478a40dd pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/c9/52/9a/71/-87fd-41a1-8b3d-077a452f8404: no such file or directory" node=c9529a71-87fd-41a1-8b3d-077a452f8404 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/14/2c/78/84/-0386-4312-843f-4101c3db44a1: no such file or directory" node=142c7884-0386-4312-843f-4101c3db44a1 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/61/76/0c/2a/-d9ba-433c-af1d-830d95c67d47: no such file or directory" node=61760c2a-d9ba-433c-af1d-830d95c67d47 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a5/05/34/9e/-de83-4190-a64e-9f94ec61306a: no such file or directory" node=a505349e-de83-4190-a64e-9f94ec61306a pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b2/3a/35/6a/-7e69-484a-ab39-fe7828cb19ce: no such file or directory" node=b23a356a-7e69-484a-ab39-fe7828cb19ce pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d1/f6/ec/01/-a5f7-4295-a3e9-8d78e8e48483: no such file or directory" node=d1f6ec01-a5f7-4295-a3e9-8d78e8e48483 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/e8/24/99/51/-449e-4a4b-84de-0110b5159447: no such file or directory" node=e8249951-449e-4a4b-84de-0110b5159447 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/35/60/83/2b/-ba10-43bb-a5ce-26d920204a36: no such file or directory" node=3560832b-ba10-43bb-a5ce-26d920204a36 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f9/94/3a/15/-847a-4aa1-bb04-be8bdabe914d: no such file or directory" node=f9943a15-847a-4aa1-bb04-be8bdabe914d pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/4f/9a/d8/b7/-63e7-4c47-806d-759984c0c406: no such file or directory" node=4f9ad8b7-63e7-4c47-806d-759984c0c406 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/95/c5/1f/c0/-cefd-4935-a085-d0af94820899: no such file or directory" node=95c51fc0-cefd-4935-a085-d0af94820899 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/95/c4/06/1c/-137b-44b4-b6a9-e35d859d8735: no such file or directory" node=95c4061c-137b-44b4-b6a9-e35d859d8735 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/64/e1/61/c5/-93ad-45dd-8fce-677ef3268ea6: no such file or directory" node=64e161c5-93ad-45dd-8fce-677ef3268ea6 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/fe/87/b5/2f/-833e-470f-84cf-a4d0a7493d32: no such file or directory" node=fe87b52f-833e-470f-84cf-a4d0a7493d32 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/3f/b0/57/00/-de89-4ed7-9a7b-8bd8a6af054f: no such file or directory" node=3fb05700-de89-4ed7-9a7b-8bd8a6af054f pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/6f/44/59/1f/-e793-4498-b380-0dcda77a7ed7: no such file or directory" node=6f44591f-e793-4498-b380-0dcda77a7ed7 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/c1/6c/0e/10/-54ab-4b6d-89fd-7405d8ef5458: no such file or directory" node=c16c0e10-54ab-4b6d-89fd-7405d8ef5458 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error listing folder error="error: not found: /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081 pkg=rgrpc service=storage-users traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:25Z ERR error purging node metadata error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081.mpk: no such file or directory" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081.mpk pkg=rgrpc service=storage-users traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:25Z ERR failed to purge recycle item error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081.mpk: no such file or directory" key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc reference={"path":".","resource_id":{"opaque_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} service=storage-users status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081.mpk: no such file or directory","trace":"2c388b5e230219e9cc5d1ccbee5a232c"} traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:25Z ERR Internal Server Error code=500 item_path=/ key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 name=com.owncloud.web.ocdav reference={"path":".","resource_id":{"space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} request-id=ca2faed6-2b2e-4462-820d-0407abc18914 service=ocdav status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081.mpk: no such file or directory","trace":"2c388b5e230219e9cc5d1ccbee5a232c"} traceid=2c388b5e230219e9cc5d1ccbee5a232c
2024-10-14T11:21:25Z ERR error listing folder error="error: not found: /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0 pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR error purging node metadata error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0.mpk: no such file or directory" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0.mpk pkg=rgrpc service=storage-users traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR failed to purge recycle item error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0.mpk: no such file or directory" key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc reference={"path":".","resource_id":{"opaque_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} service=storage-users status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0.mpk: no such file or directory","trace":"b50569db58d97eb64993797fe3cef311"} traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:25Z ERR Internal Server Error code=500 item_path=/ key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 name=com.owncloud.web.ocdav reference={"path":".","resource_id":{"space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} request-id=80a67dac-efe4-45f9-8d99-29cedb602ba3 service=ocdav status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0.mpk: no such file or directory","trace":"b50569db58d97eb64993797fe3cef311"} traceid=b50569db58d97eb64993797fe3cef311
2024-10-14T11:21:28Z WRN http end="14/Oct/2024:11:21:28 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:28 +0000" status=404 time_ns=156976 traceid=dbf164289efaa5343d04e77a549cb1c0 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b8
```

```
2024-10-14T11:21:37Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:37Z ERR error listing grantees error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node="marshaling error: json: unsupported value: encountered a cycle via *node.Node" pkg=rgrpc service=storage-users traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:37Z ERR error reading permissions error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:37Z WRN http end="14/Oct/2024:11:21:37 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:37 +0000" status=404 time_ns=113394 traceid=2acf06c10c15d907d45e40c55cdc52ee uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/e15347c1-5872-4fbf-a286-ea5d7b9008da/received.json url=/groups/e15347c1-5872-4fbf-a286-ea5d7b9008da/received.json
2024-10-14T11:21:37Z WRN http end="14/Oct/2024:11:21:37 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:37 +0000" status=404 time_ns=142579 traceid=d6d19828ea0a08086f2350e7ec4f99a9 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/users/f23e3f7b-d3a0-462f-9631-56695eabda31/received.json url=/users/f23e3f7b-d3a0-462f-9631-56695eabda31/received.json
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing grantees error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node="marshaling error: json: unsupported value: encountered a cycle via *node.Node" pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error reading permissions error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a7/fa/4e/0b/-e177-4615-87aa-54c9a2571bc0: no such file or directory" node=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/6f/44/7a/5a/-1c12-402d-8ad5-a626289c1136: no such file or directory" node=6f447a5a-1c12-402d-8ad5-a626289c1136 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/83/7b/95/83/-fcdf-4aa8-9714-ed064f7a9c57: no such file or directory" node=837b9583-fcdf-4aa8-9714-ed064f7a9c57 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/61/76/0c/2a/-d9ba-433c-af1d-830d95c67d47: no such file or directory" node=61760c2a-d9ba-433c-af1d-830d95c67d47 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/08/a6/95/d4/-fc79-4ab1-b7da-4e69478a40dd: no such file or directory" node=08a695d4-fc79-4ab1-b7da-4e69478a40dd pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/c9/52/9a/71/-87fd-41a1-8b3d-077a452f8404: no such file or directory" node=c9529a71-87fd-41a1-8b3d-077a452f8404 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/df/e2/f8/37/-572b-4d32-b3e3-18722a97546c: no such file or directory" node=dfe2f837-572b-4d32-b3e3-18722a97546c pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f2/fa/43/d6/-c890-4962-b436-31c1095c0570: no such file or directory" node=f2fa43d6-c890-4962-b436-31c1095c0570 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/14/2c/78/84/-0386-4312-843f-4101c3db44a1: no such file or directory" node=142c7884-0386-4312-843f-4101c3db44a1 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/a5/05/34/9e/-de83-4190-a64e-9f94ec61306a: no such file or directory" node=a505349e-de83-4190-a64e-9f94ec61306a pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/e8/24/99/51/-449e-4a4b-84de-0110b5159447: no such file or directory" node=e8249951-449e-4a4b-84de-0110b5159447 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/35/60/83/2b/-ba10-43bb-a5ce-26d920204a36: no such file or directory" node=3560832b-ba10-43bb-a5ce-26d920204a36 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b2/3a/35/6a/-7e69-484a-ab39-fe7828cb19ce: no such file or directory" node=b23a356a-7e69-484a-ab39-fe7828cb19ce pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d1/f6/ec/01/-a5f7-4295-a3e9-8d78e8e48483: no such file or directory" node=d1f6ec01-a5f7-4295-a3e9-8d78e8e48483 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/95/c5/1f/c0/-cefd-4935-a085-d0af94820899: no such file or directory" node=95c51fc0-cefd-4935-a085-d0af94820899 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f9/94/3a/15/-847a-4aa1-bb04-be8bdabe914d: no such file or directory" node=f9943a15-847a-4aa1-bb04-be8bdabe914d pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/fe/87/b5/2f/-833e-470f-84cf-a4d0a7493d32: no such file or directory" node=fe87b52f-833e-470f-84cf-a4d0a7493d32 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/64/e1/61/c5/-93ad-45dd-8fce-677ef3268ea6: no such file or directory" node=64e161c5-93ad-45dd-8fce-677ef3268ea6 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/95/c4/06/1c/-137b-44b4-b6a9-e35d859d8735: no such file or directory" node=95c4061c-137b-44b4-b6a9-e35d859d8735 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/c1/6c/0e/10/-54ab-4b6d-89fd-7405d8ef5458: no such file or directory" node=c16c0e10-54ab-4b6d-89fd-7405d8ef5458 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/d7/6e/ff/fe/-5ae9-4d18-b936-95409640c081: no such file or directory" node=d76efffe-5ae9-4d18-b936-95409640c081 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/4f/9a/d8/b7/-63e7-4c47-806d-759984c0c406: no such file or directory" node=4f9ad8b7-63e7-4c47-806d-759984c0c406 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/b8/34/ff/d0/-3df0-4e78-87a4-c624c7500fb0: no such file or directory" node=b834ffd0-3df0-4e78-87a4-c624c7500fb0 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/30/9e/3b/37/-2592-4618-b61c-8916db07201d: no such file or directory" node=309e3b37-2592-4618-b61c-8916db07201d pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/3f/b0/57/00/-de89-4ed7-9a7b-8bd8a6af054f: no such file or directory" node=3fb05700-de89-4ed7-9a7b-8bd8a6af054f pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/87/5b/49/8a/-926e-4f99-a96e-c3d0d1b99bda: no such file or directory" node=875b498a-926e-4f99-a96e-c3d0d1b99bda pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:38Z ERR error listing attributes error="stat /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/6f/44/59/1f/-e793-4498-b380-0dcda77a7ed7: no such file or directory" node=6f44591f-e793-4498-b380-0dcda77a7ed7 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:39Z WRN http end="14/Oct/2024:11:21:39 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:39 +0000" status=404 time_ns=139674 traceid=75d0395818151c10c542681833e13017 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json
2024-10-14T11:21:39Z WRN http end="14/Oct/2024:11:21:39 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:39 +0000" status=404 time_ns=156606 traceid=0cac25d1d32d0655c504d617213cbb9c uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json url=/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json
2024-10-14T11:21:39Z ERR error listing folder error="error: not found: /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4 pkg=rgrpc service=storage-users traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:39Z ERR error purging node metadata error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4.mpk: no such file or directory" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4.mpk pkg=rgrpc service=storage-users traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:39Z ERR failed to purge recycle item error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4.mpk: no such file or directory" key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc reference={"path":".","resource_id":{"opaque_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} service=storage-users status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4.mpk: no such file or directory","trace":"8fed4faceb7540d94a32d0282e48e159"} traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:39Z ERR Internal Server Error code=500 item_path=/ key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 name=com.owncloud.web.ocdav reference={"path":".","resource_id":{"space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} request-id=1adef68d-b6ea-42fd-9b9b-d121a4f4aeee service=ocdav status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/19/c5/26/27/-1c12-48f4-bc79-14fa105e17d4.mpk: no such file or directory","trace":"8fed4faceb7540d94a32d0282e48e159"} traceid=8fed4faceb7540d94a32d0282e48e159
2024-10-14T11:21:40Z WRN http end="14/Oct/2024:11:21:40 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:40 +0000" status=404 time_ns=144593 traceid=78036494ba62d1502a0c514de2f76cde uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json
2024-10-14T11:21:40Z WRN http end="14/Oct/2024:11:21:40 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:40 +0000" status=404 time_ns=148400 traceid=f3e4bbb3f764b49f500be267a541ea53 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json url=/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json
2024-10-14T11:21:40Z WRN http end="14/Oct/2024:11:21:40 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:40 +0000" status=404 time_ns=118424 traceid=18080b3240533150a194c7bc05a27584 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json url=/groups/1d617185-05c7-48f0-b17c-6ae84a1b875a/received.json
2024-10-14T11:21:40Z WRN http end="14/Oct/2024:11:21:40 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="14/Oct/2024:11:21:40 +0000" status=404 time_ns=145795 traceid=5fa596f3391aa7e59ce7f8f6e45cce58 uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json url=/groups/cee42433-8c74-4b57-8afc-966aeac7ddb7/received.json
2024-10-14T11:21:40Z WRN core access token not set pkg=rhttp service=frontend traceid=6e6f2f7cbe2e9ec9df421e32e48596f2
2024-10-14T11:21:40Z ERR error listing folder error="error: not found: /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8 pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:40Z ERR error purging node metadata error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8.mpk: no such file or directory" path=/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8.mpk pkg=rgrpc service=storage-users traceid=2630f0350017d96a2716221b9f376422
2024-10-14T11:21:40Z ERR failed to purge recycle item error="remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8.mpk: no such file or directory" key=a7fa4e0b-e177-4615-87aa-54c9a2571bc0 pkg=rgrpc reference={"path":".","resource_id":{"opaque_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","space_id":"5f2328fe-0a41-4de3-b210-78c95c1a896b","storage_id":"766e4e1c-7a7f-4391-b56e-46cafbcca7eb"}} service=storage-users status={"code":15,"message":"error purging recycle item:remove /var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/f4/59/2f/3c/-1ccd-4c47-bc09-4ec22f2ae8f8.mpk: no such file or directory","trace":"2630f0350017d96a2716221b9f376422"} traceid=2630f0350017d96a2716221b9f376422

2024-10-14T11:22:43Z ERR content length vs transferred bytes mismatch content-length=1516109711 pkg=rhttp request-id=6b1d0dc4-e26c-41d2-a857-71e4e99885af service=frontend traceid=1c99ae756d572e6ab4b4a3a05ea04ecb transferred-bytes=0
```

After that we tried to resynchronize the files and we ran into a runtime error.
```
{"level":"error","pid":1,"pid":1,"path":"/var/lib/ocis/storage/users/spaces/5f/2328fe-0a41-4de3-b210-78c95c1a896b/nodes/46/f7/69/7c/-2a85-4216-96c6-61de89034a87","nodeid":"46f7697c-2a85-4216-96c6-61de89034a87","attrs":{},"messagepack":"","time":"2024-10-14T11:26:59Z","caller":"github.com/cs3org/reva/v2@v2.19.7/pkg/storage/utils/decomposedfs/upload/upload.go:317","message":"missing parent id"}
2024/10/14 11:26:59 http: panic serving 127.0.0.1:49630: runtime error: invalid memory address or nil pointer dereference
goroutine 832601225 [running]:
net/http.(*conn).serve.func1()
net/http/server.go:1898 +0xbe
panic({0x4181460?, 0x6230270?})
runtime/panic.go:770 +0x132
github.com/cs3org/reva/v2/pkg/storage/utils/decomposedfs/upload.(*OcisSession).Cleanup(0xc0c1600a80, 0x1, 0x1, 0x1)
github.com/cs3org/reva/v2@v2.19.7/pkg/storage/utils/decomposedfs/upload/upload.go:319 +0xe2
github.com/cs3org/reva/v2/pkg/storage/utils/decomposedfs/upload.OcisStore.Cleanup({{0x47498c0, 0xc008104318}, {0x7f5502dbf9b0, 0xc008112500}, {0xc001a97a80, 0x1b}, {0x7f5502a74c48, 0xc007c2a2c0}, 0x1, {{0x2ada0f4, ...}, ...}}, ...)
github.com/cs3org/reva/v2@v2.19.7/pkg/storage/utils/decomposedfs/upload/store.go:161 +0x12b
github.com/cs3org/reva/v2/pkg/storage/utils/decomposedfs/upload.(*OcisSession).FinishUpload(0xc0c1600a80, {0x472d090?, 0x6538b60?})
github.com/cs3org/reva/v2@v2.19.7/pkg/storage/utils/decomposedfs/upload/upload.go:194 +0x1105
github.com/tus/tusd/pkg/handler.(*UnroutedHandler).finishUploadIfComplete(0xc0083ce1c0, {0x472d090?, 0x6538b60?}, {0x472ef30?, 0xc0c1600a80?}, {{0xc0ec9e43f0, 0x24}, 0x5a5dff8f, 0x0, 0x5a5dff8f, ...}, ...)
github.com/tus/tusd@v1.13.0/pkg/handler/unrouted_handler.go:902 +0x87
github.com/tus/tusd/pkg/handler.(*UnroutedHandler).writeChunk(0xc0083ce1c0, {0x472d090, 0x6538b60}, {0x472ef30, 0xc0c1600a80}, {{0xc0ec9e43f0, 0x24}, 0x5a5dff8f, 0x0, 0x5a5dff8f, ...}, ...)
github.com/tus/tusd@v1.13.0/pkg/handler/unrouted_handler.go:892 +0x625
github.com/tus/tusd/pkg/handler.(*UnroutedHandler).PatchFile(0xc0083ce1c0, {0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d680)
github.com/tus/tusd@v1.13.0/pkg/handler/unrouted_handler.go:758 +0x6b6
github.com/cs3org/reva/v2/pkg/rhttp/datatx/manager/tus.(*manager).Handler.func2({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d680)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/datatx/manager/tus/tus.go:181 +0x29c
net/http.HandlerFunc.ServeHTTP(0xc0a18995f0?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x2a07ce3?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/pkg/rhttp/datatx/manager/tus.(*manager).Handler.(*UnroutedHandler).Middleware.func3({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d680)
github.com/tus/tusd@v1.13.0/pkg/handler/unrouted_handler.go:296 +0x6a4
net/http.HandlerFunc.ServeHTTP(0x4150ee0?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x3?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/services/dataprovider.New.(*svc).setHandler.func1({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d680)
github.com/cs3org/reva/v2@v2.19.7/internal/http/services/dataprovider/dataprovider.go:185 +0x13e
net/http.HandlerFunc.ServeHTTP(0xc198e8d560?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x4724e50?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).registerServices.traceHandler.func1({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d560)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:316 +0x1f7
net/http.HandlerFunc.ServeHTTP(0xc287a97b07?, {0x7f5502ae9978?, 0xc0a1899740?}, 0xc14f02ad58?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).getHandler.func1({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d560)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:249 +0x1f0
net/http.HandlerFunc.ServeHTTP(0xc198e8d440?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x4724e50?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).getHandler.traceHandler.func3({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d440)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:316 +0x1f7
net/http.HandlerFunc.ServeHTTP(0xc198e8d200?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x3da6410?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/interceptors/requestid.requestIDHandler.handler-fm.requestIDHandler.handler.RequestID.func1({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8d200)
github.com/go-chi/chi/v5@v5.0.12/middleware/request_id.go:76 +0x17e
net/http.HandlerFunc.ServeHTTP(0x4724e50?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x626b290?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/interceptors/requestid.requestIDHandler.ServeHTTP(...)
github.com/cs3org/reva/v2@v2.19.7/internal/http/interceptors/requestid/requestid.go:53
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).getHandler.traceHandler.func5({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8cd80)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:316 +0x1f7
net/http.HandlerFunc.ServeHTTP(0x4710220?, {0x7f5502ae9978?, 0xc0a1899740?}, 0xc287a97b01?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/interceptors/auth.New.func2.1({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8cd80)
github.com/cs3org/reva/v2@v2.19.7/internal/http/interceptors/auth/auth.go:212 +0x7da
net/http.HandlerFunc.ServeHTTP(0xc198e8cc60?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x4724e50?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).getHandler.traceHandler.func5({0x7f5502ae9978, 0xc0a1899740}, 0xc198e8cc60)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:316 +0x1f7
net/http.HandlerFunc.ServeHTTP(0x4720a88?, {0x7f5502ae9978?, 0xc0a1899740?}, 0x0?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/interceptors/log.loggingHandler.ServeHTTP({{0x4707e28?, 0xc00812a540?}}, {0x4720a88, 0xc0c16008c0}, 0xc198e8cc60)
github.com/cs3org/reva/v2@v2.19.7/internal/http/interceptors/log/log.go:57 +0x13b
github.com/cs3org/reva/v2/pkg/rhttp.(*Server).getHandler.traceHandler.func5({0x4720a88, 0xc0c16008c0}, 0xc198e8c7e0)
github.com/cs3org/reva/v2@v2.19.7/pkg/rhttp/rhttp.go:316 +0x1f7
net/http.HandlerFunc.ServeHTTP(0x472d2c0?, {0x4720a88?, 0xc0c16008c0?}, 0xc004ee6000?)
net/http/server.go:2166 +0x29
github.com/cs3org/reva/v2/internal/http/interceptors/appctx.handler.func1({0x4720a88, 0xc0c16008c0}, 0xc198e8c6c0)
github.com/cs3org/reva/v2@v2.19.7/internal/http/interceptors/appctx/appctx.go:58 +0x6da
net/http.HandlerFunc.ServeHTTP(0x413525?, {0x4720a88?, 0xc0c16008c0?}, 0xc0c1600801?)
net/http/server.go:2166 +0x29
net/http.serverHandler.ServeHTTP({0x4717318?}, {0x4720a88?, 0xc0c16008c0?}, 0x6?)
net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc392fcbef0, {0x472d2c0, 0xc0055dcf00})
net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 557
net/http/server.go:3285 +0x4b4
2024-10-14T11:26:59Z ERR error doing PATCH request to data service error="Patch \"http://localhost:9158/data/tus/8fec9f4f-8b57-4e26-80fb-49ed1aa92d10\": EOF" pkg=rhttp request-id=61a18e0b-7616-4aa5-ace9-3e67df0a781b service=frontend traceid=fc4ec899e03ec72ac7034f59b68e0f6f
2024-10-14T11:26:59Z ERR http end="14/Oct/2024:11:26:59 +0000" host=127.0.0.1 method=PATCH pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="14/Oct/2024:11:26:50 +0000" status=500 time_ns=9218857165 traceid=fc4ec899e03ec72ac7034f59b68e0f6f uri=/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwODc0LCJpYXQiOjE3Mjg5MDQ0NzQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy84ZmVjOWY0Zi04YjU3LTRlMjYtODBmYi00OWVkMWFhOTJkMTAifQ.9zjBOPYElv9fXXgF8Kxyy6c8c7L4rMDvddbXKrZFCpc url=/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJyZXZhIiwiZXhwIjoxNzI4OTkwODc0LCJpYXQiOjE3Mjg5MDQ0NzQsInRhcmdldCI6Imh0dHA6Ly9sb2NhbGhvc3Q6OTE1OC9kYXRhL3R1cy84ZmVjOWY0Zi04YjU3LTRlMjYtODBmYi00OWVkMWFhOTJkMTAifQ.9zjBOPYElv9fXXgF8Kxyy6c8c7L4rMDvddbXKrZFCpc
2024-10-14T11:26:59Z ERR failed to authenticate the request error="token is expired" authenticator=oidc path=/dav/spaces/766e4e1c-7a7f-4391-b56e-46cafbcca7eb$5f2328fe-0a41-4de3-b210-78c95c1a896b service=proxy
```

## Steps to reproduce
Can not really give instructions to reproduce this error. We are using the instance since May this year and the problem with sync-errors is present for 1 week now. We did not delete any files manually that could lead to the error.

## Expected behavior
Uploads shouldn't fail and deletions should go through.

## Actual behavior
Upload / sync is failing and we receive a runtime error while deleting files.

## Setup
Please describe how you started the server and provide a list of relevant environment variables or configuration files.

No special configuration. Using Version `5.0.8` as of yesterday with OIDC enabled and S3 as storage backend.