Query with subscribe enabled makes request after signing out

I have a query that lists all documents in a collection that I subscribe on and if I sign out I keep getting a snapshot listener error because I have a rule that only allows the documents to be accessed if signed in. The component where the query is located gets unmounted properly and doesn't re-render after signing out. I have also tried to remove the query using the query client before signing out but that doesn't seem to remove the snapshot listener but it does stop the realtime changes from working.