dotnet / dotnet/orleans

Unit Testing & OrleansTestKit Path Forward

Open
#4,534 2 comments 0 reactions 0 assignees View on GitHub
design-review help wanted
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
14h 42m
Merged PRs (30d)
354

Description

We have been discussing our testing strategy internally and would like to see if anyone else has opinions on how we can make testing better in Orleans. We are thinking about using a similar API to the one I created in OrleansTestKit, but run it within the actual Orleans runtime. The OrleansTestKit is able to run tests for many simple use cases, however it will not support more complex grain that are deeply integrated with Orleans for example, Lifecycle events. There are many other things that Orleans does that would not be able to be tested inside the TestKit. Rather than spending more time emulating Orleans behaviors, we are thinking about a new approach that would allow the same assertions that exist within the current TestKit, but would allow grain code to execute exactly as it does when running in a real silo.

In reality, most of this proposal is just to allow developers to assert grain state, grain calls to other grains, and provide a way for a developer configure a mock grain that will return the desired value or exception when a grain is called.

What would need to be built:

**Storage provider**
- Assert the current grain state
- Setup the grain state prior to activation

**Stream provider**
- Assert that a grain is subscribed to a specific stream
- Assert that a grain sends the proper message on a stream
- Assert that a grain received a message from a stream without error
- Assert that a grain unsubscribed from a specific stream

**Reminder Provider**
- Assert that a reminder was registered for the correct time
- Fire a reminder on demand from within a test

**Mock Grain Call Filter**
- Assert that another grain was called with the correct values
- Intercept grain-to-grain method calls and return a specific value or throw an exception

**Other**
- Deactivate a grain on demand
- Activate a grain on demand

Please let me know if you have other things that you would like exposed and we can talk about a roadmap to build out these features. We are going to start prototyping some of these behaviors this week to see how they work in practice.

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by reviewing the existing OrleansTestKit and the proposed runtime-based testing approach, then evaluate the listed storage, stream, reminder, mock grain call filter, activation, and deactivation behaviors. Done would require an agreed roadmap and working prototypes, but the issue does not define a bounded first change.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
distributed-systems, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.