Cross references duplicated and don't display correctly when switching ILs

Version and Platform (required):

• Binary Ninja Version: 4.1.5746
• OS: Ubuntu
• OS Version: 24.04
• CPU Architecture: aarch64

Bug Description:
We now have what appears to be weirdly duplicated cross-references when switching ILs in certain cases? I'm pretty sure that's not actually what's happening, it's just what the effect looks like from a user perspective.

Here's an example in HLIL on 4.0 stable:

When you switch from HLIL to disassembly, it won't change at first, and then will eventually change to this, which is also wrong:

Re-clicking on that address leads to this:

In the new 4.1 stable we're testing, it does this in HLIL:

...which immediately switches to this:

...and clicking on the address again gives this:

Steps To Reproduce:

1. Open the binary below
2. Navigate to 0x217af4
3. Click on x0
4. Observe cross-references
5. Change view to Disassembly
6. Observe cross-references again

Expected Behavior:
Something closer to what's in 4.0 than 4.1. And, also, showing what I see when I re-click on the address possibly.

Screenshots/Video Recording:
See above.

Binary:
libicui18n.so.70.zip

Open the supplied libicui18n.so.70 binary in Binary Ninja and reproduce the sequence at 0x217af4: click x0, inspect cross-references, switch from HLIL to Disassembly, and click the address again. Trace the cross-reference display and IL-switch handling; done means references are not duplicated or stale and remain consistent across the views.