[BUG]: Can't deploy to databricks anymore

**Describe the bug**
A clear and concise description of what the bug is.

dotnet spark jobs do not run on the latest version of databricks clusters in Azure.
This worked before (about 3 or so months ago) but is no longer working.
With the db_init.sh (see attached) based on the one given in the repository there is no log output of any kind.
Without db_init.sh there is an error in the stderr log:

```scala
Caused by: java.lang.NullPointerException
at com.databricks.backend.daemon.driver.WsfsDriverHttpClient.(WSFSDriverHttpClient.scala:26)
at com.databricks.backend.daemon.driver.WSFSCredentialForwardingHelper$.(WorkspaceLocalFileSystem.scala:284)
at com.databricks.backend.daemon.driver.WSFSCredentialForwardingHelper$.(WorkspaceLocalFileSystem.scala)
... 28 more
```

Steps to reproduce the behavior:
1. create a simple dotnet spark job that reads a csv file and shows the results
2. publish the job
3. deploy the job to databricks cluser
4. run the job
5. note the job fails and there is no log output of any kind

**Expected behavior**
The job should not fail
[db_init.zip](https://github.com/dotnet/spark/files/8919459/db_init.zip)