Simplify testing infrastructure to use Testcontainers
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 13h 56m
- Merged PRs (30d)
- 351
Description
## Goal
Simplify the testing infrastructure by migrating from GitHub Actions services to [Testcontainers](https://testcontainers.com/), which will make tests easier to run locally and maintain.
## Current State
The CI workflow currently uses GitHub Actions services for various providers. Here are the Docker images being used:
### Database Providers
- **Redis**: `redis` (latest)
- **PostgreSQL**: `postgres` (latest)
- **MariaDB/MySQL**: `mariadb:10.6`
- **SQL Server**: `mcr.microsoft.com/mssql/server:latest`
### Azure Services
- **Azure Storage (Azurite)**: `mcr.microsoft.com/azure-storage/azurite:latest`
### Clustering Providers
- **ZooKeeper**: `zookeeper:3.9`
- **Consul**: (uses local installation, not containerized yet)
### AWS Providers
- **DynamoDB**: `amazon/dynamodb-local:latest`
### Streaming Providers
- **NATS**: `nats:latest` (started with `--js --http-port=8222` flags)
### Not Containerized
- **Cassandra**: Currently uses environment variable `CASSANDRAVERSION` with versions 4.0, 4.1, 5.0 (needs investigation on how it's started)
- **Cosmos DB**: Currently disabled (`if: ${{ false }}`) - uses Azure Cosmos DB Emulator on Windows
## Tasks
- [ ] Migrate Redis tests to Testcontainers
- [ ] Migrate PostgreSQL tests to Testcontainers
- [ ] Migrate MariaDB/MySQL tests to Testcontainers
- [ ] Migrate SQL Server tests to Testcontainers
- [ ] Migrate Azure Storage (Azurite) tests to Testcontainers #9921
- [ ] Migrate ZooKeeper tests to Testcontainers
- [x] Migrate Consul tests to Testcontainers #9707
- [ ] Migrate DynamoDB tests to Testcontainers
- [ ] Migrate NATS tests to Testcontainers
- [ ] Migrate Cassandra tests to Testcontainers (versions 4.0, 4.1, 5.0)
- [ ] Investigate and migrate Cosmos DB tests to Testcontainers (if feasible)
- [ ] Update documentation for running tests locally
- [ ] Remove GitHub Actions services configuration from CI workflow
## Migration Plan
For each provider, we should:
1. Replace GitHub Actions `services` with Testcontainers equivalents
2. Ensure tests can run both in CI and locally
3. Maintain the same test coverage and configuration
4. Update documentation for running tests locally
## Benefits
- **Local Development**: Developers can run all provider tests locally without manual setup
- **Consistency**: Same container images used in CI and local development
- **Maintainability**: Easier to update versions and configurations
- **Cross-platform**: Works on Windows, Linux, and macOS (where Docker is available)
Contributor guide
Research direction
Start by locating the CI workflow's services configuration and the test setup for one unchecked provider, then compare how that provider is started in CI. Migrate one provider while preserving its existing coverage and configuration, verify it runs locally and in CI, and update the local-testing documentation before removing the corresponding service entry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions
- Domain
- ci-cd, infrastructure, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100