litestar-org / litestar-org/pytest-databases
Add message broker / emulator fixtures: Kafka (Redpanda), LocalStack, Google Pub/Sub emulator, RabbitMQ
- Dominant language
- Python
- Stars
- 59
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Add Docker service fixtures for message brokers and cloud emulators, following the same `*_service` + Docker-autoskip pattern as the existing fixtures (the Spanner emulator fixture is the closest precedent):
1. **Kafka** — `kafka_service` exposing bootstrap servers. Candidate images: `redpandadata/redpanda` in dev-container mode (Kafka API-compatible, ~2x faster startup, no ZooKeeper/KRaft ceremony) or `apache/kafka-native` (KRaft, GraalVM native image). Could also ship both (`kafka_service` / `redpanda_service`).
2. **LocalStack** — `localstack_service` exposing `endpoint_url`, region, and test credentials, so downstream tests can exercise SQS (visibility timeouts, DelaySeconds, FIFO) and other AWS APIs against `localstack/localstack`.
3. **Google Pub/Sub emulator** — `pubsub_service` mirroring the Spanner emulator fixture shape (project, host/port, `PUBSUB_EMULATOR_HOST`-style wiring), e.g. `gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators` running `gcloud beta emulators pubsub start`.
4. **RabbitMQ** — `rabbitmq_service` exposing host/port/user/password (and optionally the management API port), backed by the official `rabbitmq:4-management` image. No emulator needed — it's the real broker; RabbitMQ 4.3+ is preferable so downstream tests can cover quorum-queue strict priority and native delayed messages.
## Motivation
litestar-queues is researching broker-style integrations (Kafka / Google Pub/Sub / Amazon SQS / RabbitMQ). Its integration matrix consumes pytest-databases `*_service` fixtures with importorskip + Docker autoskip gating, and the project policy is to add missing container fixtures upstream here rather than hand-roll session-scoped fixtures downstream. These fixtures would also be broadly useful beyond that use case.
Happy to split this into one issue per fixture if preferred.
Contributor guide
Research direction
Start by reading the existing Spanner emulator fixture and the other *_service fixtures that use Docker autoskip and importorskip gating. Decide whether to split the four services, then add fixtures exposing the connection details requested for Kafka, LocalStack, Pub/Sub, and RabbitMQ. Done means downstream integration tests can consume each fixture without hand-rolled session fixtures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, google-cloud, kafka, python, rabbitmq
- Domain
- cloud, devops, distributed-systems, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100