Azure / Azure/azure-sdk-tools

[Stress] Validating chaos deployment

Open
#5,908 0 comments 0 reactions 1 assignee Claimed by @ckairen View on GitHub
Central-EngSys Stress
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
144

Description

We want to validate that the chaos-mesh policies are activating when they should and that they are properly "chaos-ing" the correct resources.

@lmolkova found this recently where we were using different naming conventions for created resources vs blocked host names, which meant the chaos-mesh policy wasn't in effect for the resources that we use. We also ran into an issue where our chaos policies weren't activating due to some missing attributes when targeting pods.

These are mostly config, and somewhat loose so nothing fails in an obvious way. We can add in some simple validation to help us here:

1. Add in validation to make sure that the templates all line up properly on values. We do some of this already to make sure you're not referencing an old version of our Helm templates, so this could be an expansion of that.
2. Create a CI job that deploys resources and a test with 100% network loss so we can tell if things are properly blocked.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.