Enhancement: HTTP/2, HTTP/3, and WebSocket Fallback for Faster Streaming

Enhancement: HTTP/2, HTTP/3, and WebSocket Fallback for Faster Streaming

Summary

Add HTTP/2, HTTP/3, and WebSocket fallback support to the proxy architecture for improved streaming performance, reduced latency, and better reliability for unreliable connections.

Motivation

Current proxy implementation uses HTTP/1.1 with Server-Sent Events (SSE) for streaming. This works well but has limitations:

• HTTP/1.1 Limitations: Head-of-line blocking, no multiplexing, higher latency
• SSE Reliability: Can fail on unreliable connections (mobile, poor WiFi)
• Performance Gap: Missing 30-50% performance gains from HTTP/2+ multiplexing
• Mobile Use Cases: WebSocket fallback needed for unstable connections

Benefits

HTTP/2 Support

• ✅ Multiplexing - Multiple streams over single connection (30-50% faster)
• ✅ Header compression - HPACK reduces overhead by 30-80%
• ✅ Server push - Proactive data delivery
• ✅ Stream prioritization - Critical responses first
• ✅ Binary protocol - More efficient than text-based HTTP/1.1

HTTP/3 Support (QUIC)

• ✅ Zero RTT - Faster connection establishment (50-70% faster than HTTP/2)
• ✅ No head-of-line blocking - Independent streams
• ✅ Better mobile - Handles network switches gracefully
• ✅ Built-in encryption - TLS 1.3 mandatory
• ✅ Already have QUIC transport - Can leverage existing implementation

WebSocket Fallback

• ✅ Bidirectional - Full-duplex communication
• ✅ Mobile-friendly - Better for unstable connections
• ✅ Lower overhead - No HTTP headers per message
• ✅ Reconnection - Automatic retry on disconnect
• ✅ Universal support - Works everywhere (browsers, mobile, desktop)

Proposed Implementation

Phase 1: HTTP/2 Support (Estimated: 2-3 days)

Dependencies:

{
  "spdy": "^4.0.2",  // HTTP/2 server (Node.js)
  "http2": "built-in" // Node.js native module
}

Implementation:

1. Create HTTP/2 Proxy Server (src/proxy/http2-proxy.ts)

import http2 from 'http2';
import { readFileSync } from 'fs';

export class HTTP2Proxy {
  private server: http2.Http2SecureServer;

  constructor(config: {
    cert: string;
    key: string;
    port: number;
  }) {
    this.server = http2.createSecureServer({
      cert: readFileSync(config.cert),
      key: readFileSync(config.key),
      allowHTTP1: true // Fallback to HTTP/1.1
    });

    this.setupRoutes();
  }

  private setupRoutes() {
    this.server.on('stream', (stream, headers) => {
      const path = headers[':path'];

      if (path === '/v1/messages') {
        this.handleStreamingRequest(stream, headers);
      } else {
        stream.respond({ ':status': 404 });
        stream.end('Not Found');
      }
    });
  }

  private async handleStreamingRequest(
    stream: http2.ServerHttp2Stream,
    headers: http2.IncomingHttpHeaders
  ) {
    try {
      // Read request body
      const chunks: Buffer[] = [];
      stream.on('data', (chunk) => chunks.push(chunk));

      await new Promise((resolve) => stream.on('end', resolve));
      const body = JSON.parse(Buffer.concat(chunks).toString());

      // Convert and forward to provider
      const geminiReq = this.convertAnthropicToGemini(body);

      // Stream response
      stream.respond({
        ':status': 200,
        'content-type': 'text/event-stream',
        'cache-control': 'no-cache',
        'connection': 'keep-alive'
      });

      // Forward to Gemini and stream back
      const response = await fetch(geminiUrl, {
        method: 'POST',
        body: JSON.stringify(geminiReq)
      });

      const reader = response.body?.getReader();
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const anthropicChunk = this.convertGeminiStreamToAnthropic(value);
        stream.write(anthropicChunk);
      }

      stream.end();

    } catch (error) {
      stream.respond({ ':status': 500 });
      stream.end(JSON.stringify({ error: error.message }));
    }
  }

  start(port: number) {
    this.server.listen(port, () => {
      console.log(`HTTP/2 proxy running at https://localhost:${port}`);
    });
  }
}

2. Add HTTP/2 CLI Option

// src/cli-proxy.ts
if (options.http2) {
  const http2Proxy = new HTTP2Proxy({
    cert: options.cert || './certs/cert.pem',
    key: options.key || './certs/key.pem',
    port: options.port || 3000
  });
  await http2Proxy.start(options.port);
}

3. Performance Benchmark

// validation/benchmark-http2.ts
async function benchmarkHTTP2vsHTTP1() {
  const results = {
    http1: await measureLatency('http://localhost:3000'),
    http2: await measureLatency('https://localhost:3001') // HTTP/2
  };

  console.log(`HTTP/1.1 avg latency: ${results.http1.avg}ms`);
  console.log(`HTTP/2 avg latency: ${results.http2.avg}ms`);
  console.log(`Improvement: ${((1 - results.http2.avg / results.http1.avg) * 100).toFixed(1)}%`);
}

Expected Performance:

• 30-50% faster streaming latency
• 30-80% header compression savings
• Better concurrent request handling

---

Phase 2: HTTP/3 (QUIC) Support (Estimated: 3-4 days)

Dependencies:

{
  "@fails-components/webtransport": "^0.2.1", // HTTP/3 WebTransport
  "node-quic": "^0.4.0" // QUIC protocol
}

Note: We already have QUIC transport in src/transport/quic.ts - can leverage this!

Implementation:

1. Extend Existing QUIC Transport (src/transport/quic-proxy.ts)

import { QuicTransport } from './quic.js';

export class HTTP3Proxy {
  private transport: QuicTransport;

  constructor(config: {
    port: number;
    cert: string;
    key: string;
  }) {
    this.transport = new QuicTransport({
      host: 'localhost',
      port: config.port,
      cert: config.cert,
      key: config.key,
      alpn: ['h3'], // HTTP/3 ALPN
      maxConcurrentStreams: 100
    });
  }

  async start() {
    await this.transport.listen();

    this.transport.on('stream', async (stream) => {
      const headers = await stream.readHeaders();

      if (headers[':path'] === '/v1/messages') {
        await this.handleLLMRequest(stream, headers);
      }
    });

    console.log('HTTP/3 proxy running on QUIC transport');
  }

  private async handleLLMRequest(stream: any, headers: any) {
    // Read request body from QUIC stream
    const body = await stream.read();
    const anthropicReq = JSON.parse(body);

    // Convert and forward
    const geminiReq = this.convertAnthropicToGemini(anthropicReq);

    // Send response headers
    await stream.writeHeaders({
      ':status': '200',
      'content-type': 'text/event-stream'
    });

    // Stream response chunks
    const reader = await this.fetchGemini(geminiReq);
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = this.convertGeminiStreamToAnthropic(value);
      await stream.write(chunk);
    }

    await stream.end();
  }
}

2. Add HTTP/3 CLI Option

// src/cli-proxy.ts
if (options.http3 || options.quic) {
  const http3Proxy = new HTTP3Proxy({
    port: options.port || 4433,
    cert: options.cert || './certs/cert.pem',
    key: options.key || './certs/key.pem'
  });
  await http3Proxy.start();
}

3. Performance Benchmark

// validation/benchmark-http3.ts
async function benchmarkHTTP3() {
  const results = {
    http1: await measureLatency('http://localhost:3000'),
    http2: await measureLatency('https://localhost:3001'),
    http3: await measureLatency('https://localhost:4433') // HTTP/3
  };

  console.log('Protocol Performance Comparison:');
  console.log(`HTTP/1.1: ${results.http1.avg}ms (baseline)`);
  console.log(`HTTP/2:   ${results.http2.avg}ms (${improvement(results.http1, results.http2)}%)`);
  console.log(`HTTP/3:   ${results.http3.avg}ms (${improvement(results.http1, results.http3)}%)`);
}

Expected Performance:

• 50-70% faster than HTTP/2 (zero RTT)
• No head-of-line blocking
• Better mobile/WiFi performance

---

Phase 3: WebSocket Fallback (Estimated: 2-3 days)

Dependencies:

{
  "ws": "^8.18.3" // Already installed!
}

Implementation:

1. Create WebSocket Proxy (src/proxy/websocket-proxy.ts)

import { WebSocketServer, WebSocket } from 'ws';
import { createServer } from 'http';

export class WebSocketProxy {
  private wss: WebSocketServer;
  private server: any;

  constructor(config: { port: number }) {
    this.server = createServer();
    this.wss = new WebSocketServer({ server: this.server });
    this.setupHandlers();
  }

  private setupHandlers() {
    this.wss.on('connection', (ws: WebSocket) => {
      console.log('WebSocket client connected');

      ws.on('message', async (data: Buffer) => {
        try {
          const message = JSON.parse(data.toString());

          if (message.type === 'streaming_request') {
            await this.handleStreamingRequest(ws, message.data);
          }

        } catch (error) {
          ws.send(JSON.stringify({
            type: 'error',
            error: error.message
          }));
        }
      });

      ws.on('close', () => {
        console.log('WebSocket client disconnected');
      });

      ws.on('error', (error) => {
        console.error('WebSocket error:', error);
      });

      // Send initial handshake
      ws.send(JSON.stringify({
        type: 'connected',
        protocols: ['anthropic-messages-v1']
      }));
    });
  }

  private async handleStreamingRequest(
    ws: WebSocket,
    anthropicReq: any
  ) {
    // Convert to Gemini format
    const geminiReq = this.convertAnthropicToGemini(anthropicReq);

    // Send streaming start event
    ws.send(JSON.stringify({
      type: 'message_start',
      message: { id: `msg_${Date.now()}`, role: 'assistant' }
    }));

    // Forward to Gemini and stream back
    const response = await fetch(geminiUrl, {
      method: 'POST',
      body: JSON.stringify(geminiReq)
    });

    const reader = response.body?.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const anthropicChunk = this.convertGeminiStreamToAnthropic(chunk);

      // Send each chunk as WebSocket message
      ws.send(JSON.stringify({
        type: 'content_block_delta',
        delta: { type: 'text_delta', text: anthropicChunk }
      }));
    }

    // Send completion
    ws.send(JSON.stringify({
      type: 'message_stop'
    }));
  }

  start(port: number) {
    this.server.listen(port, () => {
      console.log(`WebSocket proxy running at ws://localhost:${port}`);
    });
  }
}

2. Auto-Fallback Logic (src/proxy/adaptive-proxy.ts)

export class AdaptiveProxy {
  async start(config: {
    enableHTTP2: boolean;
    enableHTTP3: boolean;
    enableWebSocket: boolean;
  }) {
    const servers = [];

    // Try HTTP/3 first (fastest)
    if (config.enableHTTP3) {
      try {
        const http3 = new HTTP3Proxy({ port: 4433 });
        await http3.start();
        servers.push({ protocol: 'HTTP/3', port: 4433 });
      } catch (error) {
        console.warn('HTTP/3 unavailable, falling back to HTTP/2');
      }
    }

    // Try HTTP/2 next
    if (config.enableHTTP2) {
      try {
        const http2 = new HTTP2Proxy({ port: 3001 });
        await http2.start();
        servers.push({ protocol: 'HTTP/2', port: 3001 });
      } catch (error) {
        console.warn('HTTP/2 unavailable, falling back to HTTP/1.1');
      }
    }

    // HTTP/1.1 (always available)
    const http1 = new AnthropicToGeminiProxy({ port: 3000 });
    await http1.start();
    servers.push({ protocol: 'HTTP/1.1', port: 3000 });

    // WebSocket fallback for unreliable connections
    if (config.enableWebSocket) {
      const ws = new WebSocketProxy({ port: 8080 });
      await ws.start();
      servers.push({ protocol: 'WebSocket', port: 8080 });
    }

    console.log('\n✅ Multi-Protocol Proxy Started:');
    servers.forEach(s => {
      console.log(`  ${s.protocol.padEnd(12)} → Port ${s.port}`);
    });

    return servers;
  }
}

3. Client-Side Auto-Detection

// Client automatically picks best protocol
export async function createOptimalConnection(baseUrl: string) {
  // Try HTTP/3 first
  try {
    const http3Client = new HTTP3Client(`${baseUrl}:4433`);
    await http3Client.ping();
    return http3Client;
  } catch {}

  // Try HTTP/2
  try {
    const http2Client = new HTTP2Client(`${baseUrl}:3001`);
    await http2Client.ping();
    return http2Client;
  } catch {}

  // Try WebSocket for unreliable connections
  try {
    const wsClient = new WebSocketClient(`${baseUrl}:8080`);
    await wsClient.connect();
    return wsClient;
  } catch {}

  // Fallback to HTTP/1.1
  return new HTTP1Client(`${baseUrl}:3000`);
}

---

Configuration

New CLI Flags:

# HTTP/2 support
npx agentic-flow proxy --http2 --cert ./certs/cert.pem --key ./certs/key.pem

# HTTP/3 (QUIC) support
npx agentic-flow proxy --http3 --port 4433

# WebSocket fallback
npx agentic-flow proxy --websocket --port 8080

# Multi-protocol (all enabled)
npx agentic-flow proxy --all-protocols

Environment Variables:

ENABLE_HTTP2=true
ENABLE_HTTP3=true
ENABLE_WEBSOCKET=true
HTTP2_PORT=3001
HTTP3_PORT=4433
WEBSOCKET_PORT=8080

---

Performance Targets

Protocol	Target Latency	Multiplexing	Mobile Support
HTTP/1.1	Baseline (100ms)	❌ No	⚠️ Limited
HTTP/2	30-50% faster (50-70ms)	✅ Yes	⚠️ Limited
HTTP/3	50-70% faster (30-50ms)	✅ Yes	✅ Excellent
WebSocket	Variable (depends on connection)	✅ Yes	✅ Excellent

---

Testing Strategy

1. Unit Tests - Each protocol handler
2. Integration Tests - End-to-end streaming
3. Performance Benchmarks - Latency, throughput, concurrency
4. Mobile Testing - Unstable connections, network switches
5. Load Testing - 100+ concurrent connections
6. Fallback Testing - Protocol negotiation, automatic downgrade

---

Migration Path

Phase 1: Add HTTP/2 (Week 1)

• Implement HTTP/2 proxy
• Add CLI flag
• Benchmark performance
• Document usage

Phase 2: Add HTTP/3 (Week 2)

• Extend QUIC transport
• Integrate with existing QUIC code
• Benchmark vs HTTP/2
• Mobile testing

Phase 3: Add WebSocket (Week 3)

• Implement WebSocket proxy
• Auto-fallback logic
• Reliability testing
• Mobile app integration

Phase 4: Production (Week 4)

• Multi-protocol support
• Auto-detection
• Load balancing
• Monitoring & metrics

---

Success Metrics

• ✅ 30-50% latency improvement with HTTP/2
• ✅ 50-70% latency improvement with HTTP/3
• ✅ 99.9% uptime with WebSocket fallback
• ✅ Zero config - automatic protocol selection
• ✅ Mobile-first - seamless network switching

---

References

• HTTP/2 Spec: https://httpwg.org/specs/rfc7540.html
• HTTP/3 Spec: https://datatracker.ietf.org/doc/html/rfc9114
• QUIC Protocol: https://datatracker.ietf.org/doc/html/rfc9000
• WebSocket API: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket
• Existing QUIC Transport: /workspaces/agentic-flow/agentic-flow/src/transport/quic.ts

---

Related Issues

• #50 - Gemini provider fixes (RESOLVED in v1.9.3)
• #51 - Provider fallback system (RESOLVED in v1.9.4)

---

Labels

enhancement, performance, streaming, http2, http3, websocket, proxy, good-first-issue (for HTTP/2 implementation)

Start by reviewing src/cli-proxy.ts and the existing QUIC transport in src/transport/quic.ts, then compare the proposed entry points in src/proxy/http2-proxy.ts, src/transport/quic-proxy.ts, src/proxy/websocket-proxy.ts, and src/proxy/adaptive-proxy.ts. Check the validation benchmark files described for HTTP/2 and HTTP/3. Done would require the proposed protocol support, fallback behavior, and performance validation across the listed phases.