OAuth2AuthorizedClient doesn't get removed when 403 returned by Resource Server

Describe the bug
when configuring WebClient using ServerOAuth2AuthorizedClientExchangeFilterFunction with AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager, if the resource server returns 403, the OAuth2AuthorizedClient doesn't get removed.

To Reproduce

@Bean
public WebClient oauth2WebClient(
      final WebClient.Builder webClientBuilder,
      final ReactiveClientRegistrationRepository registrationRepository,
      final ReactiveOAuth2AuthorizedClientService authorizedClientService,
      final String clientRegistrationId) {

    final ServerOAuth2AuthorizedClientExchangeFilterFunction secureExchangeFilterFunction =
        new ServerOAuth2AuthorizedClientExchangeFilterFunction(
            new AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(
                registrationRepository, authorizedClientService));
    
    secureExchangeFilterFunction.setAuthorizationFailureHandler(
        new RemoveAuthorizedClientReactiveOAuth2AuthorizationFailureHandler(
            (clientRegistrationId, principal, attributes) ->
                authorizedClientService.removeAuthorizedClient(
                    clientRegistrationId, principal.getName())
        ));
    secureExchangeFilterFunction.setDefaultClientRegistrationId(clientRegistrationId);

    return webClientBuilder.clone().filter(secureExchangeFilterFunction).build();
}

when using the above oauth2WebClient to make a call to a resource server, the resource server returns 403.
A subsequence call to the resource server uses the old access token.

Expected behavior
when using the above oauth2WebClient to make a call to a resource server, the resource server returns 403.
A subsequence call to the resource server should retrieve a new access token from the authorization server.

Initial thoughts

RemoveAuthorizedClientReactiveOAuth2AuthorizationFailureHandler is initialized with DEFAULT_REMOVE_AUTHORIZED_CLIENT_ERROR_CODES which contains only INVALID_TOKEN and INVALID_GRANT. However, with 403 returned by the resource server, ServerOAuth2AuthorizedClientExchangeFilterFunction#AuthorizationFailureForwarder maps 403 to INSUFFICIENT_SCOPE. This causes condition hasRemovalErrorCode() in RemoveAuthorizedClientReactiveOAuth2AuthorizationFailureHandler#onAuthorizationFailure not satisfy hence Oauth2AuthorizedClient doesn't get removed.

https://github.com/spring-projects/spring-security/blob/1ff5eb6b57d2ba55c82a90c2150aa760c36ec237/oauth2/oauth2-client/src/main/java/org/springframework/security/oauth2/client/RemoveAuthorizedClientReactiveOAuth2AuthorizationFailureHandler.java#L113-L120

Work Around

@Bean
public WebClient oauth2WebClient(
      final WebClient.Builder webClientBuilder,
      final ReactiveClientRegistrationRepository registrationRepository,
      final ReactiveOAuth2AuthorizedClientService authorizedClientService,
      final String clientRegistrationId) {

    final ServerOAuth2AuthorizedClientExchangeFilterFunction secureExchangeFilterFunction =
        new ServerOAuth2AuthorizedClientExchangeFilterFunction(
            new AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(
                registrationRepository, authorizedClientService));
    
    Set<String> removeAuthorizedClientErrorCodes =
        new HashSet<>(DEFAULT_REMOVE_AUTHORIZED_CLIENT_ERROR_CODES);
    removeAuthorizedClientErrorCodes.add(INSUFFICIENT_SCOPE); // 403

    secureExchangeFilterFunction.setAuthorizationFailureHandler(
        new RemoveAuthorizedClientReactiveOAuth2AuthorizationFailureHandler(
            (clientRegistrationId, principal, attributes) ->
                authorizedClientService.removeAuthorizedClient(
                    clientRegistrationId, principal.getName()),
             removeAuthorizedClientErrorCodes
        ));
    secureExchangeFilterFunction.setDefaultClientRegistrationId(clientRegistrationId);

    return webClientBuilder.clone().filter(secureExchangeFilterFunction).build();
}