SimpleX uses previously configured SMP servers when custom SMP server is entered for previously created contacts

Im actually not entirely sure if this is an issue or my fault.
Sry in advance, if i just didnt understand something crucial.

So i've setup a SMP Server on Ubuntu 2204 LTS using the latest binary,
like its described in the Documentation, and in General this all seems to be fine:

Jul 25 18:41:58 systemd[1]: Started Simplex SMP Server.
Jul 25 18:41:58 smp-server[2068]: SMP server v3.1.2
Jul 25 18:41:58 smp-server[2068]: Fingerprint: (removed, not for the public....)
Jul 25 18:41:58 smp-server[2068]: Store log disabled.
Jul 25 18:41:58 smp-server[2068]: Listening on port 5223 (TLS)...
Jul 25 18:41:58 smp-server[2068]: not expiring inactive clients
Jul 25 18:41:58 smp-server[2068]: server stats log enabled: /var/opt/simplex/smp-server-stats.daily.log

I entered the SMP server in my terminal Client, as well as in my Android App using smp://@:5223, ofc replaced with my individual fingerprint and ip.
Both clients accepted it.

Now i wanted to test if the messages are delivered over my server,
so i turned off the Terminal Cient, and sent a Message from my Android Device.
Then i turned off Simplex Chat on Android in the Settings, and enabled my Terminal Client,
and reveived the message, so it must have passed some server, as no direct connection should have been possible. Alright.

Then i tried same again, but turned off my SMP Server before, and miraculously the messages still arrived, without P2P connection, and SMP Server turned off? So i thought there must be some fallback, even if you have customized a own SMP, is that right?
How can i ensure, that my own SMP Server ist used? (at least if both chat partners customized the same one) Or did i just get wrong, how this whole System works?

I also tried it, with having my phone in the mobile network, and Terminal Application in my home wifi, getting the same result.

Thanks in advance for any ideas :)

Edit:
I know this is supposed to be decentralized, but at least i thought it would dispatch my messages over my server if i customize one...

Start with the documented SMP server setup and reproduce the route using the terminal client, Android app, and the custom smp:// endpoint described here, with the server stopped. Done means establishing whether previously configured contacts intentionally use fallback SMP servers or incorrectly bypass the custom server, then recording the resolution in the issue or documentation.