mcp: Implement Automatic OAuth Callback Handling with Local Server

Scope

For the Authorization Code flow, automate the callback capture using a temporary local HTTP server that listens on the redirect_uri, eliminating the need for manual copy-paste in most scenarios.

Haystack MCP Integration Details

Callback Server Design:

• Use Python's http.server or starlette for lightweight local server
• Server should bind to the redirect_uri host/port (must be localhost/127.0.0.1)
• Server should shut down cleanly after receiving callback
• Handle timeout (e.g., 5 minutes) if no callback received

Browser Integration:

• Use webbrowser module to automatically open authorization URL
• Gracefully handle cases where browser cannot be opened (headless environments)
• Display success message in browser after callback capture

Fallback Strategy:

• If server cannot bind (port in use, permissions) → fallback to manual paste
• If no callback received within timeout → fallback to manual paste
• If running in headless environment → fallback to manual paste
• Keep on_authorization_url callback hook customizable for advanced use cases

Integration with OAuth Flow:

• Callback server should be started before redirecting user to authorization URL
• Server should extract code and state from query parameters
• Verify state parameter matches the one sent (CSRF protection)
• Return extracted code to OAuth flow for token exchange
• Display friendly success message in browser

Requirements

• Implement local HTTP server that binds to redirect_uri (if localhost)
• Automatically open system browser to authorization URL (when possible)
• Capture callback request and extract code and state parameters
• Verify state parameter for CSRF protection
• Display success message in browser (HTML response)
• Shut down server cleanly after callback
• Fallback to manual paste if:
  • Server cannot bind (port in use, permissions)
  • No callback received within timeout (e.g., 5 minutes)
  • Running in headless environment
  • Browser cannot be opened
• Make on_authorization_url callback hook still customizable for advanced use cases
• Handle random port selection if redirect_uri port is 0 or unspecified
• Add tests for automatic callback capture (mock server)
• Add tests for fallback scenarios
• Add tests for state verification

Implementation Notes

• Consider using threading or asyncio for server (must work in sync context)
• Server should be single-request (shutdown after first callback)
• Ensure proper error handling and cleanup

Start by locating the existing OAuth Authorization Code flow and its on_authorization_url callback hook, then trace the current manual-paste path. Implement and test local callback capture, state verification, browser handling, cleanup, timeout, and fallback behavior, including mock-server and fallback tests described in the issue.