No Native HTTP Retry Mechanism in Apps-Engine

No Native HTTP Retry Mechanism in Apps-Engine

Current Situation

Currently, the Rocket.Chat Apps-Engine lacks a native retry mechanism for HTTP requests. This means that apps need to implement their own retry logic when dealing with unreliable external services or temporary network issues, leading to:

• Duplicate code across apps
• Inconsistent retry implementations
• No standardized way to handle transient failures

Proposed Solution

Implement a new RetryHttpProvider class that can be optionally enabled by apps to provide robust retry functionality without affecting existing apps or adding overhead to the base implementation.

Implementation Details

1. Create RetryHttpProvider Class

export class RetryHttpProvider extends Http {
    private static readonly ENHANCED_RETRY_CONFIG: IHttpRetryConfig = {
        enabled: true,
        maxAttempts: 3,
        initialDelay: 1000,
        statusCodesToRetry: [
            HttpStatusCode.SERVICE_UNAVAILABLE,
            HttpStatusCode.INTERNAL_SERVER_ERROR,
            HttpStatusCode.TOO_MANY_REQUESTS,
            HttpStatusCode.GATEWAY_TIMEOUT,
            HttpStatusCode.BAD_GATEWAY
        ],
    };
    // Inherit from base Http class to maintain all existing functionality
    constructor(
        accessManager: AppAccessorManager,
        bridges: AppBridges,
        httpExtender: IHttpExtend,
        appId: string,
    ) {
        super(accessManager, bridges, httpExtender, appId);
    }

    private enhanceOptions(options?: IHttpRequest): IHttpRequest {
        return {
            ...options,
            retry: {
                ...RetryHttpProvider.ENHANCED_RETRY_CONFIG,
                ...options?.retry,
            },
        };
    }
    // Override HTTP methods to include retry functionality
    public get(url: string, options?: IHttpRequest): Promise<IHttpResponse> {
        return super.get(url, this.enhanceOptions(options));
    }
   // ... other HTTP methods
}

2. Create Configuration Manager

export class RetryHttpConfiguration {
  public async setup(configuration: IConfigurationExtend): Promise<void> {
    // Add settings for retry configuration
    await Promise.all([
 configuration.settings.provideSetting({
      id: 'enable-http-retries',
      type: SettingType.BOOLEAN,
      packageValue: false,
      required: true,
      public: true,
      i18nLabel: 'enable-http-retries',
    });
  configuration.settings.provideSetting({
          id: `${this.config.alias}-max-attempts`,
          type: SettingType.NUMBER,
          public: true,
          required: true,
          packageValue: '',
          i18nLabel: `${this.config.alias}-max-attempts`,
      }),

      configuration.settings.provideSetting({
          id: `${this.config.alias}-initial-delay`,
          type: SettingType.NUMBER,
          public: true,
          required: true,
          packageValue: '',
          i18nLabel: `${this.config.alias}-initial-delay`,
      }),

      configuration.settings.provideSetting({
        id: `${this.config.alias}-status-codes-to-retry`,
        type: SettingType.MULTI_SELECT,
        public: true,
        required: true,
        packageValue: '',
        i18nLabel: `${this.config.alias}-status-codes-to-retry`,
    }),
  }
}

3. Usage in Apps

export class YourApp implements IApp {
  private retryConfig: RetryHttpConfiguration;
  protected async extendConfiguration(configuration: IConfigurationExtend): Promise<void> {
    await this.retryConfig.setup(configuration);
}
}

Benefits

1. Zero Overhead for Existing Apps

• Existing apps continue to use the base Http class
• No performance impact on apps not using retry functionality
• No breaking changes to existing implementations

2. Opt-in Enhancement

• Apps can explicitly opt-in to retry functionality by using RetryHttpConfiguration
• Configuration is handled during app initialization
• Settings are exposed in the admin UI for easy configuration

3. Consistent Implementation

• Standardized retry behavior across all apps using the feature
• Configurable retry parameters (attempts, delay, status codes)
• Built on top of existing HTTP infrastructure

Implementation Steps

1. Core Changes

   • Add IHttpRetryConfig interface to define retry configuration
   • Implement RetryHttpProvider class extending base Http
   • Create RetryHttpConfiguration for settings management

2. Accessor Manager Modifications
   This is a critical part of the implementation that enables the override functionality:

   export class AppAccessorManager {
       private httpProviders: Map<string, IHttp>;
   
       constructor(
           private readonly bridges: AppBridges,
           private readonly httpExtender: IHttpExtend,
       ) {
           this.httpProviders = new Map();
       }
   
       public getHttp(appId: string): IHttp {
           if (!this.httpProviders.has(appId)) {
               // Check if the app has enabled retry functionality
               const settings = this.getAppSettings(appId);
               if (settings.get('enable-http-retries')) {
                   this.httpProviders.set(
                       appId,
                       new RetryHttpProvider(this, this.bridges, this.httpExtender, appId)
                   );
               } else {
                   // Use default Http implementation for apps without retry enabled
                   this.httpProviders.set(
                       appId,
                       new Http(this, this.bridges, this.httpExtender, appId)
                   );
               }
           }
   
           return this.httpProviders.get(appId);
       }
   }
   

   How the Override Works

   1. The AppAccessorManager is responsible for providing HTTP instances to apps
   2. When an app requests an HTTP instance via getHttp():
      • Checks if the app has enabled retry functionality in its settings
      • If enabled, provides a RetryHttpProvider instance
      • If not enabled, provides the default Http instance
   3. The instance is cached in httpProviders map for subsequent requests
   4. All HTTP calls from the app will use the provided instance

   This approach ensures:

   • Zero overhead for apps not using retry functionality
   • Automatic retry capability for apps that enable it
   • Clean separation of concerns
   • No breaking changes to existing apps

3. Integration

   • Add retry configuration settings to app configuration system
   • Implement factory pattern in AppAccessorManager for HTTP provider creation
   • Add documentation and examples

4. Testing

   • Unit tests for retry logic
   • Integration tests with various status codes
   • Performance testing to ensure no impact on non-retry apps

Migration Guide

For apps wanting to use the retry functionality:

1. Import the RetryHttpConfiguration
2. Add it to your app's configuration setup:

protected async extendConfiguration(configuration: IConfigurationExtend): Promise<void> {
const retryConfig = new RetryHttpConfiguration();
await retryConfig.setup(configuration);
}

Server Setup Information:

• Version of Rocket.Chat Server:
• License Type:
• Number of Users:
• Operating System:
• Deployment Method:
• Number of Running Instances:
• DB Replicaset Oplog:
• NodeJS Version:
• MongoDB Version:

Client Setup Information

• Desktop App or Browser Version:
• Operating System:

Additional context

Relevant logs:

Start with packages/apps-engine/src/server/managers/AppAccessorManager.ts and trace getHttp() into the existing Http provider and HTTP interfaces. Review the issue's proposed retry configuration and provider behavior, then define the required unit, integration, and performance tests. Done means opt-in retry support works without changing existing apps or their default HTTP behavior.