PublishDir step fails on push to S3 even though pipeline succeeds

Bug report

Reported by a user.

I recently observed a run on Nextflow Tower that was reported to be completed successfully on platform, but the publish directory on S3 was empty.
The corresponding directory in the working directory was fully populated with all expected outputs.

Log file showed the following:

Nov-16 07:14:35.678 [PublishDir-1] WARN  nextflow.processor.PublishDir - Failed to publish file: s3://s3-path/scratch/2NQLCVMB9up84P/dd/be6eea71f1d896340271b30c751e39/results; to: s3://s3-path/users/kkosheleva/tower/EXP23002963/MAX078/results [copy] -- See log file for details
java.nio.file.NoSuchFileException: the path: /s3-path/users/kkosheleva/tower/EXP23002963/MAX078/results/variants not exists
    at com.upplication.s3fs.S3FileSystemProvider.delete(S3FileSystemProvider.java:561)
    at java.base/java.nio.file.Files.delete(Files.java:1152)
    at nextflow.file.FileHelper$3.postVisitDirectory(FileHelper.groovy:1007)
    at nextflow.file.FileHelper$3.postVisitDirectory(FileHelper.groovy)
    at java.base/java.nio.file.Files.walkFileTree(Files.java:2828)
    at java.base/java.nio.file.Files.walkFileTree(Files.java:2882)
    at nextflow.file.FileHelper.deleteDir0(FileHelper.groovy:999)
    at nextflow.file.FileHelper.deletePath(FileHelper.groovy:991)
    at nextflow.processor.PublishDir.processFile(PublishDir.groovy:390)
    at nextflow.processor.PublishDir.safeProcessFile(PublishDir.groovy:359)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1258)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
    at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:1036)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1019)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:97)
    at nextflow.processor.PublishDir$_apply1_closure1.doCall(PublishDir.groovy:332)
    at nextflow.processor.PublishDir$_apply1_closure1.call(PublishDir.groovy)
    at groovy.lang.Closure.run(Closure.java:493)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Nov-16 07:14:35.679 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false)