Plugin submission: Bright Data Web Intelligence Processor

Summary

We've built a Python Dynamic Plugin for Redpanda Connect that integrates
Bright Data's web intelligence APIs as native pipeline processors.

What it does

The plugin exposes four Bright Data operations as a single brightdata
processor:

• Web Unlocker — fetch any webpage, bypassing anti-bot protection
• SERP — Google search results retrieval
• Web Scraper — structured data extraction from pre-built scrapers (e.g. Amazon, LinkedIn)
• Dataset — query Bright Data's maintained datasets

Architecture

• Python Dynamic Plugin (gRPC subprocess), built on the redpanda-connect
  Python SDK
• Runs in Redpanda Connect's streams mode, supporting multiple simultaneous
  pipelines
• Async operations (Scraper, Dataset) handle trigger → poll → retrieve
  automatically
• Built-in retry logic with exponential backoff

Status

• Fully implemented and integration tested against the live Bright Data API
• Demoed internally and to Bright Data, who have approved moving forward
• Repository: https://github.com/ReallyGreatTech/brightdata-redpanda

What we need from the Redpanda team

We'd like guidance on:

1. The certification process and requirements for community plugins,
   specifically Python Dynamic Plugins
2. Whether Python Dynamic Plugins follow the same certification path as
   Go-native connectors described in CONTRIBUTING.md, or a different process
3. Where in the repo structure / documentation this should be referenced
4. Any naming conventions or manifest standards we should follow
5. Review process and timeline expectations

Happy to share the full functional/technical specs for review.

Thanks!

Start with CONTRIBUTING.md and the linked brightdata-redpanda repository. Review the existing certification guidance for Go-native connectors, then determine how Python Dynamic Plugins, repository placement, naming, manifests, and review expectations should be documented. Done means the Redpanda team has provided a clear submission path or requested concrete materials for review.