Panic in `write_plt_entry_bytes:278`

I'm getting a panic and the stacktrace points at the line shown below. This happens at the very early stages when I initialize my JIT and try to declare some functions. Any hints on how to solve this?

```
unsafe fn write_plt_entry_bytes(plt_ptr: *mut [u8; 16], got_ptr: NonNull>) {
...................
plt_val[2..6].copy_from_slice(&i32::to_ne_bytes(i32::try_from(what - at).unwrap()));
...................
}
```

### More info

- I'm running on `x86_64` and `Ubuntu 20`.
- My JIT only declares some external rust functions and creates a wrapper function that calls those.
- I previously ran into another memory execution permissions problem and was pointed to use the feature flag `selinux-fix` which solved that problem (see https://github.com/bytecodealliance/wasmtime/issues/4980#issue-1391090347).
- Disabling `is_pic` obviously solves this panic, is that a bug?