[BUG] Request to DO gives 'TypeError: Fetch API cannot load'

### Is there an existing issue for this?

- [X] I have searched the existing issues

### What version of `workers-rs` are you using?

0.0.18

### What version of `wrangler` are you using?

3.5.0

### Describe the bug

I created the simplest possible worker that fetches from a durable object and every request from worker to durable object fails with:

```
Error: TypeError: Fetch API cannot load: /get_cached - Cause: TypeError: Fetch API cannot load: /get_cached
```

I verified that he durable object is available and that the error happens at `fetch_with_str`. I also put the durable object into a separate module. I have billing for durable objects enabled. Here's the reproducer: https://github.com/buchgr/workers-repro

### Steps To Reproduce

```
$ git clone https://github.com/buchgr/workers-repro.git
$ npx wrangler dev
```
[b] open a browser
```
Error: TypeError: Fetch API cannot load: /get_cached - Cause: TypeError: Fetch API cannot load: /get_cached
```