HLS failure with GHC 9.14 and ExplicitLevelImports

I turned `LANGUAGE ExplicitLevelImports` ON on my codebase (1200+ haskell modules) and got it building with GHC 9.14.

However HLS fails on some modules (repro is not obvious) with:

```
Diagnostics:
Compilation Issue
expectJust

Call stack:
CallStack (from HasCallStack):
pprPanic, called at compiler/GHC/Data/Maybe.hs:77:19 in ghc-9.14.1-05a0:GHC.Data.Maybe
expectJustError, called at compiler/GHC/Data/Maybe.hs:74:24 in ghc-9.14.1-05a0:GHC.Data.Maybe
expectJust, called at compiler/GHC/Unit/Module/Graph.hs:579:8 in ghc-9.14.1-05a0:GHC.Unit.Module.Graph
Please report this bug to the compiler authors.
```

I bet that this is linked to the custom graph logic that HLS does for template haskell code. Maybe this custom logic could be simplified in the context of `ExplicitLevelImports` (e.g. GHC claims to be better at tracking dependencies for template haskell).

### Your environment

Which OS do you use? Nixos
Which version of GHC do you use and how did you install it? GHC 9.14 from nixpkgs
How is your project built (alternative: link to the project)? Custom nix builder

Which LSP client (editor/plugin) do you use? neovim
Which version of HLS do you use and how did you install it? 2.13.0.0+some custom patchs
Have you configured HLS in any way (especially: a `hie.yaml` file)? using cmd line flag

### Steps to reproduce

Sorry, the following is left empty until I find a proper way to reproduce. It does seem to only appear on module which depends themselfs on module which does have template haskell.

### Expected behaviour

HLS should typecheck correctly.

### Actual behaviour

HLS fails

### Debug information