App Service Authentication with multiple identity providers causing slow HTTP request performance

We are running a function app (consumption plan) with app service authentication enabled. Adding one identity provider does not decrease HTTP request performance. Adding a second identity provider decreases request performance from about 100 ms to 1s-2s. We are using two OpenID Connect configurations against a Azure B2C directory and a Azure Active directory, by passing the OpenID Connect metadata document endpoint. Deleting one of the identity providers increases the request performance.

The App Service Authentication with these configurations work like expected, but is causing a bad request response time.