Track Orleans provider integration and end-to-end testing with Aspire
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 14h 42m
- Merged PRs (30d)
- 354
Description
## Goal
Provide a complete, tested Orleans + Aspire experience for every backend which has both:
1. an Orleans provider in this repository; and
2. a first-party Aspire hosting integration maintained by Microsoft or the service owner.
Each supported pairing should produce valid Orleans configuration from an Aspire AppHost and have tests which prove both configuration activation and live provider behavior.
## Why
`Aspire.Hosting.Orleans` can model clustering, grain storage, reminders, streaming, and grain directories, but support is only reliable when the Orleans provider assembly registers an `IProviderBuilder` for the provider type and correctly consumes the configuration emitted by the Aspire resource reference.
The repository currently has many provider builders but very little `DistributedApplicationTestingBuilder` coverage. Some capabilities have no builder, some resource types need explicit provider-type overrides, and grain journaling is not currently modeled by `Aspire.Hosting.Orleans`. Unit tests which configure a provider directly do not prove that the AppHost resource model, generated environment, keyed client registration, configuration binding, and Orleans startup work together.
PR #9328 and issue #9244 establish the Event Hubs pattern. Aspire tests should be added to each provider's existing test project rather than creating separate `*.Aspire.Tests` projects.
## Scope and inventory
The initial scope uses first-party Aspire integrations from `microsoft/aspire` plus the AWS-maintained `Aspire.Hosting.AWS` integration. Community-only/manual-container integrations are listed separately below.
| Backend | Aspire integration | Orleans capabilities | Current Orleans configuration status | Tracking work |
|---|---|---|---|---|
| Azure Storage Tables/Blobs/Queues | `Aspire.Hosting.Azure.Storage` and Aspire Azure client integrations | Clustering, table/blob grain storage, reminders, grain directory, Azure Queue streaming, table/blob journaling | Provider builders exist for all listed Orleans packages; app-model/live Aspire coverage is missing. `Aspire.Hosting.Orleans` does not model grain journaling. | Add end-to-end Aspire tests to existing Azure provider test projects; add/coordinate a journaling resource-model API or document explicit configuration. |
| Azure Cosmos DB | `Aspire.Hosting.Azure.CosmosDB` | Clustering, grain storage, reminders | Builders exist for clustering and grain storage; Cosmos reminders lacks configuration-driven provider registration. | Add the reminders builder and end-to-end emulator tests in existing Cosmos/Azure test projects. |
| Redis and Azure Managed Redis | `Aspire.Hosting.Redis` / `Aspire.Hosting.Azure.Redis` | Clustering, grain storage, reminders, grain directory, streaming, journaling | Builders exist for all capabilities; AppHost-to-live-provider tests are missing. `Aspire.Hosting.Orleans` does not model grain journaling. | Add Aspire model and live Redis tests in `Orleans.Redis.Tests`; coordinate journaling model support. |
| SQL Server | `Aspire.Hosting.SqlServer` / `Aspire.Hosting.Azure.Sql` | ADO.NET clustering, grain storage, reminders, grain directory, streaming | Builders exist for clustering, storage, reminders, and grain directory; ADO.NET streaming lacks a provider builder. Aspire docs currently require manual ADO.NET configuration. | Add database-resource provider inference/override guidance, streaming builder, schema provisioning, and existing-project integration tests. |
| PostgreSQL | `Aspire.Hosting.PostgreSQL` / `Aspire.Hosting.Azure.PostgreSQL` | ADO.NET clustering, grain storage, reminders, grain directory, streaming | Same shared ADO.NET builder gaps as SQL Server. | Add AppHost and live provider tests in existing ADO.NET tests using the PostgreSQL CI service. |
| MySQL/MariaDB | `Aspire.Hosting.MySql` | ADO.NET clustering, grain storage, reminders, grain directory, streaming | Same shared ADO.NET builder gaps as SQL Server. | Add AppHost and live provider tests in existing ADO.NET tests using the MySQL CI service. |
| Oracle Database | `Aspire.Hosting.Oracle` | ADO.NET clustering, grain storage, reminders | Shared ADO.NET builders exist; provider invariant and Aspire connection-property mapping need explicit verification. Orleans has no Oracle grain-directory or streaming schema. | Add model/activation tests and a gated live Oracle test where CI licensing/runtime constraints permit it. |
| Azure Event Hubs | `Aspire.Hosting.Azure.EventHubs` | Streaming with Azure Table checkpoints | Active work in #9244 / #9328. | Complete the provider builder and place Aspire tests in the existing Event Hubs test project, including emulator-backed publish/consume/checkpoint behavior. |
| NATS | `Aspire.Hosting.Nats` | NATS JetStream streaming | No configuration-driven NATS stream provider builder. | Add silo/client provider builders plus AppHost and live JetStream tests in the existing NATS test project. |
| Amazon DynamoDB | AWS-maintained `Aspire.Hosting.AWS` (`AddDynamoDBTable`, DynamoDB Local, CloudFormation/CDK) | Clustering, grain storage, reminders | Provider builders exist; Aspire model/live tests are missing and configuration mapping needs an audit. | Add tests to `Orleans.AWS.Tests`, using DynamoDB Local in the existing AWS CI job and model-only tests for CDK/CloudFormation outputs. |
| Amazon SQS | AWS-maintained `Aspire.Hosting.AWS` (`AddSQSQueue`, CloudFormation/CDK) | Streaming, including FIFO and data adapters | No SQS stream provider builder; one Aspire queue does not directly represent Orleans' partitioned queue topology. | #10783 |
| Amazon Kinesis | AWS-maintained `Aspire.Hosting.AWS` (`AddKinesisStream`, CloudFormation/CDK) | Rewindable streaming with DynamoDB or grain checkpoints | No Kinesis stream provider builder. | Add resource/configuration design, silo/client builders, checkpoint wiring, and tests in `Orleans.AWS.Tests`; use model tests plus the existing Kinesis/DynamoDB CI services where practical. |
Official integration references:
- Aspire integration source: https://github.com/microsoft/aspire/tree/main/src
- Aspire Orleans integration: https://aspire.dev/integrations/frameworks/orleans/
- AWS Aspire integration: https://github.com/aws/integrations-on-dotnet-aspire-for-aws
## Standard implementation contract
For every row:
- Register stable Orleans provider aliases using `RegisterProvider` for every applicable capability and host role (silo/client).
- Implement `IProviderBuilder` and, where applicable, `IProviderBuilder`.
- Accept the configuration shape emitted by the Aspire resource, including `ServiceKey`, `ConnectionName`, direct connection values, and resource-specific connection properties.
- Prefer keyed Aspire clients when the Orleans provider uses the service SDK directly; use injected connection strings for ADO.NET providers.
- Support `WithOrleansProviderType(...)` when Aspire cannot infer the Orleans provider name from the resource type.
- Validate missing, ambiguous, or incompatible configuration during startup with actionable errors.
- Preserve secret redaction and workload/managed identity behavior.
- Document provider-specific topology and lifecycle ownership: which resources Aspire provisions, which resources Orleans creates, and which must exist before startup.
- Update the GitHub Pages Aspire provider matrix and add compiled AppHost/silo/client examples.
## Standard test contract
Place Aspire tests in the existing provider test project. Do not add one Aspire-only test project per provider.
Each provider should have:
1. **App-model tests** using `DistributedApplicationTestingBuilder`:
- create the backing Aspire resource and Orleans resource;
- connect them using `WithClustering`, `WithGrainStorage`, `WithReminders`, `WithStreaming`, or `WithGrainDirectory`;
- evaluate generated environment/configuration;
- build a silo/client host from that configuration;
- verify provider activation and bound options;
- cover provider-type inference and explicit `WithOrleansProviderType` overrides;
- cover invalid/incomplete configuration.
2. **Live integration tests** using the provider's existing emulator/container/CI service:
- start a cluster from Aspire-generated configuration;
- exercise the provider's real capability (membership, state round trip, reminders, directory lookup, stream publish/consume/checkpoint, or journaling recovery);
- verify restart/recovery behavior where that is a core provider guarantee;
- run under the provider's existing GitHub Actions job and traits.
3. **Framework coverage**:
- keep Aspire app-model tests on the framework required by the current Aspire packages;
- exercise the resulting Orleans provider behavior on every Orleans target framework supported by the existing provider CI job (`net8.0` and `net10.0` today) where package constraints permit it.
## Cross-repository Aspire gaps
Open or link upstream Aspire issues when the Orleans integration cannot represent a capability correctly. Known examples:
- `WithStreaming` and other provider operations currently require `IResourceWithConnectionString`; AWS CDK constructs expose structured outputs instead.
- `Aspire.Hosting.Orleans` does not currently expose a grain-journaling operation.
- Some services represent one physical resource while an Orleans provider owns a partitioned resource set.
- ADO.NET resource types do not infer the `AdoNet` provider name or invariant automatically.
## Excluded from the initial matrix
Orleans also supports Cassandra, Consul, ZooKeeper, and Google Cloud Firestore. They do not currently have first-party Aspire hosting integrations in `microsoft/aspire` or a service-owner Aspire integration equivalent to `Aspire.Hosting.AWS`. Track them here only after adopting an official integration contract; community containers can still be documented as manual AppHost resources.
SQLite is also excluded because it is an embedded/local database rather than a distributed Aspire backing-service resource.
## Completion checklist
- [ ] Azure Storage capabilities have app-model and live Aspire tests.
- [ ] Cosmos DB clustering, storage, and reminders have complete provider builders and Aspire tests.
- [ ] Redis/Azure Redis capabilities have app-model and live Aspire tests.
- [ ] ADO.NET support is proven for SQL Server, PostgreSQL, MySQL/MariaDB, and Oracle; ADO.NET streaming has a provider builder.
- [ ] Event Hubs work in #9244 / #9328 is complete and tests are in the existing Event Hubs test project.
- [ ] NATS streaming has provider builders and Aspire tests.
- [ ] DynamoDB clustering/storage/reminders have AWS Aspire tests.
- [ ] SQS work in #10783 is complete.
- [ ] Kinesis streaming has AWS Aspire integration and tests.
- [ ] Grain journaling resource modeling is implemented upstream or has a documented, tested explicit configuration path.
- [ ] The GitHub Pages Aspire integration matrix reflects every verified pairing and links to provider-specific guidance.
Contributor guide
Research direction
Start with the existing provider test projects and the Event Hubs pattern established by PR #9328 and issue #9244. Use DistributedApplicationTestingBuilder with Aspire.Hosting.Orleans to map each inventory row to app-model and live integration coverage. Done means the completion checklist is satisfied, including provider activation, real capability tests, supported target frameworks, and the updated GitHub Pages matrix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, azure, csharp, mysql, postgresql, redis, sql
- Domain
- backend, cloud, databases, distributed-systems, documentation, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100