issue while running on azure HDinsight cluster

- **A description of the bug**

I am trying out a simple dask example on azure hdinsight cluster but application fails after submitting and gives below stack trace.

- **Steps to reproduce**

I am executing given code using python3.5 from master node (SSHed into it).
I have archived venv using venv-pack and uploaded it on azure distributed/wasb storage.

`
from dask_yarn import YarnCluster;
from dask.distributed import Client;
cluster =
YarnCluster(environment='wasb:///user/sshuser/dask_dedup.tar.gz',worker_vcores=2,worker_memory="8GiB",n_workers=2)`

- **Relevant logs/tracebacks**
20/11/13 07:36:32 INFO skein.ApplicationMaster: Starting Skein version 0.8.0
20/11/13 07:36:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20/11/13 07:36:33 INFO skein.ApplicationMaster: Running as user sshuser
20/11/13 07:36:33 INFO conf.Configuration: found resource resource-types.xml at file:/etc/hadoop/3.1.2.7-1/0/resource-types.xml
20/11/13 07:36:33 INFO skein.ApplicationMaster: Application specification successfully loaded
20/11/13 07:36:33 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2-azure-file-system.properties
20/11/13 07:36:33 INFO sink.WasbAzureIaasSink: Init starting.
20/11/13 07:36:33 INFO sink.AzureIaasSink: Init starting. Initializing MdsLogger.
20/11/13 07:36:33 INFO sink.AzureIaasSink: Init completed.
20/11/13 07:36:33 INFO sink.WasbAzureIaasSink: Init completed.
20/11/13 07:36:33 INFO impl.MetricsSinkAdapter: Sink azurefs2 started
20/11/13 07:36:33 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 60 second(s).
20/11/13 07:36:33 INFO impl.MetricsSystemImpl: azure-file-system metrics system started
20/11/13 07:36:33 INFO client.RequestHedgingRMFailoverProxyProvider: Created wrapped proxy for [rm1, rm2]
20/11/13 07:36:34 INFO skein.ApplicationMaster: gRPC server started at
20/11/13 07:36:34 INFO skein.ApplicationMaster: WebUI server started at
20/11/13 07:36:34 INFO skein.ApplicationMaster: Registering application with resource manager
20/11/13 07:36:34 INFO client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]...
20/11/13 07:36:34 INFO client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm2]
20/11/13 07:36:34 INFO client.RequestHedgingRMFailoverProxyProvider: Created wrapped proxy for [rm1, rm2]
20/11/13 07:36:34 INFO client.AHSProxy: Connecting to Application History server at headnodehost
20/11/13 07:36:34 INFO client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]...
20/11/13 07:36:34 INFO client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm2]
20/11/13 07:36:34 INFO skein.ApplicationMaster: Initializing service 'dask.worker'.
20/11/13 07:36:34 INFO skein.ApplicationMaster: WAITING: dask.worker_0
20/11/13 07:36:34 INFO skein.ApplicationMaster: WAITING: dask.worker_1
20/11/13 07:36:34 INFO skein.ApplicationMaster: Initializing service 'dask.scheduler'.
20/11/13 07:36:34 INFO skein.ApplicationMaster: REQUESTED: dask.scheduler_0
20/11/13 07:36:35 INFO skein.ApplicationMaster: Starting container_e10_1599254613788_0080_01_000002...
20/11/13 07:36:35 INFO skein.ApplicationMaster: RUNNING: dask.scheduler_0 on container_e10_1599254613788_0080_01_000002
20/11/13 07:36:35 INFO skein.ApplicationMaster: REQUESTED: dask.worker_0
20/11/13 07:36:35 INFO skein.ApplicationMaster: REQUESTED: dask.worker_1
20/11/13 07:36:37 INFO skein.ApplicationMaster: Starting container_e10_1599254613788_0080_01_000003...
20/11/13 07:36:37 INFO skein.ApplicationMaster: RUNNING: dask.worker_0 on container_e10_1599254613788_0080_01_000003
20/11/13 07:36:37 INFO skein.ApplicationMaster: Starting container_e10_1599254613788_0080_01_000004...
20/11/13 07:36:37 INFO skein.ApplicationMaster: RUNNING: dask.worker_1 on container_e10_1599254613788_0080_01_000004
20/11/13 07:36:42 WARN skein.ApplicationMaster: FAILED: dask.worker_1 - [2020-11-13 07:36:41.666]wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/.skein.pem: No such file or directory.
java.io.FileNotFoundException: wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/.skein.pem: No such file or directory.
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatusInternal(NativeAzureFileSystem.java:2715)
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus(NativeAzureFileSystem.java:2619)
at org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:269)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

20/11/13 07:36:42 INFO skein.ApplicationMaster: RESTARTING: adding new container to replace dask.worker_1.
20/11/13 07:36:42 INFO skein.ApplicationMaster: REQUESTED: dask.worker_2
20/11/13 07:36:42 WARN skein.ApplicationMaster: FAILED: dask.worker_0 - [2020-11-13 07:36:37.992]wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/dask.worker.sh: No such file or directory.
java.io.FileNotFoundException: wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/dask.worker.sh: No such file or directory.
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatusInternal(NativeAzureFileSystem.java:2715)
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus(NativeAzureFileSystem.java:2619)
at org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:269)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

20/11/13 07:36:42 INFO skein.ApplicationMaster: RESTARTING: adding new container to replace dask.worker_0.
20/11/13 07:36:42 INFO skein.ApplicationMaster: REQUESTED: dask.worker_3
20/11/13 07:36:42 WARN skein.ApplicationMaster: FAILED: dask.scheduler_0 - [2020-11-13 07:36:40.532]wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/dask.scheduler.sh: No such file or directory.
java.io.FileNotFoundException: wasb://dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080/dask.scheduler.sh: No such file or directory.
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatusInternal(NativeAzureFileSystem.java:2715)
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.getFileStatus(NativeAzureFileSystem.java:2619)
at org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:269)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

20/11/13 07:36:42 INFO skein.ApplicationMaster: Shutting down: Failure in service dask.scheduler, see logs for more information.
20/11/13 07:36:42 INFO skein.ApplicationMaster: Unregistering application with status FAILED
20/11/13 07:36:42 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered.
20/11/13 07:36:43 WARN azure.AzureFileSystemThreadPoolExecutor: Disabling threads for Delete operation as thread count 0 is <= 1
20/11/13 07:36:43 INFO azure.AzureFileSystemThreadPoolExecutor: Time taken for Delete operation is: 58 ms with threads: 0
20/11/13 07:36:43 INFO skein.ApplicationMaster: Deleted application directory wasb://tech-dsl-hdi-spark-dev-2020-06-15t09-47-36-041z@dslhdisparkdehdistorage1.blob.core.windows.net/user/sshuser/.skein/application_1599254613788_0080
20/11/13 07:36:43 INFO skein.ApplicationMaster: WebUI server shut down
20/11/13 07:36:43 INFO skein.ApplicationMaster: gRPC server shut down
20/11/13 07:36:43 INFO impl.MetricsSystemImpl: Stopping azure-file-system metrics system...
20/11/13 07:36:43 INFO impl.MetricsSinkAdapter: azurefs2 thread interrupted.
20/11/13 07:36:43 INFO impl.MetricsSystemImpl: azure-file-system metrics system stopped.
20/11/13 07:36:43 INFO impl.MetricsSystemImpl: azure-file-system metrics system shutdown complete.

- **Version information**

Please include version information for the following:

- Python version : 3.5
- Dask-Yarn version : 0.8.1
- Hadoop version: Hadoop 3.1.1.3.1.2.7-1