Allow some user to be denied from linking their account to a specific OAuth source

Hi, I have a feature suggestion for OAuth Source and Federation !

**Is your feature request related to a problem? Please describe.**
I want to use a OAuth source to allow user to connect to my authentik from that other OAuth Source. But I want that only specific user (or no user) can link their account to this OAuth source.

**Describe the solution you'd like**
One solution could be to add a field in a group or user (or other object) to list allowed or unallowed OAuth sources linkage for this object.

**Describe alternatives you've considered**
An other option could be to prevent all linkage for a specific OAuth source but it could restrain some configurations.
The first solution could be implemented directly in the OAuth source instead of in the objects.

**Additional context**
My use case is two Authentik instances where one is the instance for infrastructure administration and the other is the instance for users. The infrastructure account should be able to connect to the user instance to manage it. But the admin shouldn't be able to link their user account to their admin account.

Thanks for your help !