Model 'distiluse-base-multilingual-cased' couldn't be loaded if CUDA is not enabled

I'm trying to work with the multilingual model - 'distiluse-base-multilingual-cased'. I need to load it on CPU-only devices. But I can't load it there because I've got an error:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False...

(the full message can be found in GIST with simple code for reproducing)

I am working in the Windows environment + Anaconda, Python 3.7. But the same behavior can be repeated in Colab with using CPU-only runtime.

Please, find minimal code for replication of this problem in gist https://gist.github.com/serge-sotnyk/bb031b622f2e9b27cb466897f7b61315. Here you could find the full error message with the stack trace.

If you uncomment the following line:
```
model = SentenceTransformer('bert-base-nli-mean-tokens')
```
and comment
```
model = SentenceTransformer('distiluse-base-multilingual-cased')
```
all works, but I need the multilingual model...

P.S.: It seems to me, this issue can be connected with https://github.com/explosion/spacy-transformers/issues/33.

Start with the minimal reproduction in the linked Gist and the SentenceTransformer model-loading entry point, using a CPU-only runtime. Compare loading distiluse-base-multilingual-cased with bert-base-nli-mean-tokens; done means the multilingual model loads without attempting CUDA deserialization.

Written by the indexing model from the issue text.