Azure / Azure/bicep

Bicep Experimental Test Framework

Open
#11,967 18 comments 15 reactions 0 assignees View on GitHub
enhancement story: test-framework
Dominant language
Bicep
Stars
3.6k
Forks
830
Avg merge
1d 21m
Merged PRs (30d)
79

Description

# Bicep Experimental Test Framework
Owner: Emily Redmond

## Introduction

We began prototyping experimental test features early in summer 2023, and we’ve been encouraged by the level of customer interest in the project. While we are still early in exploring what the future of testing looks like in Bicep, we are eager to get early feedback. If you choose to enable the feature, beware the functionality is still **experimental and subject to change at any time**.

Read the full proposal for the testing framework [here](https://github.com/Azure/bicep/issues/11966).

## How to get started with experimental test framework
1. Enable `assertions` and `testFramework` experimental feature flags in a `bicepconfig.json` file. Both feature flags must be enabled for expected functionality.
2. Create a new `.bicep` file to author your tests, eg. `tests.bicep`. This must be a separate file from the `.bicep` template you want to test. Author a test block with the new `test` keyword, referencing a `.bicep` template you would like to test on the client-side (without deploying).
3. Author `assert` statements to validate parameters, variables, and resource names within the .bicep template file you would like to test that you referenced in your test blocks.
4. In your CLI, run the command `bicep test ` to run `test` blocks and their corresponding `assert` statements without deploying.

## 1. Enable experimental features
### Enable testing experimental features in `bicepconfig.json`
1. Create `bicepconfig.json` file within your Bicep project repo

![image](https://user-images.githubusercontent.com/69916790/267428657-b0a8ac43-28a9-4045-ad9d-16e3973abfc0.png)

2. Within `bicepconfig.json`, define `experimentalFeaturesEnabled` and set `testFramework: true` and `assertions: true`

![image](https://user-images.githubusercontent.com/69916790/267429349-2081fa3b-fa56-4e9f-9696-42254313a303.png)

## 2. Test blocks
### `test` block syntax:
- `test` keyword
- Name of test block (eg. `testMain`)
- Reference to Bicep file to test (eg. `‘main.bicep’`)
- `params` object mocking parameter values from the Bicep file (eg. `params: {env: 'prod'}`)

### Example:

![image](https://user-images.githubusercontent.com/69916790/267427810-ee90b31f-ff81-4123-a4c4-70a5aacc8f21.png)

### Lmitations:
- You must pass in a parameter object, you cannot reference an existing parameter file currently
- Test blocks must be in a separate file from the .bicep template you want to test
- Test result output summary is not standardized

## 3. Assert statements
### `assert` statement syntax:
- `assert` keyword
- Name of assert statement (eg. `appName`)
- equals **boolean** validation expression (eg. `= contains(appServiceApp.name, "${env}"`)

`assert` statements can be any boolean expression that references parameters, variables, or resource names. These boolean expressions can include Bicep functions such as "contains()", "length()", etc.

### Examples:

![image](https://user-images.githubusercontent.com/69916790/268119244-c3d2c285-44e0-42a3-8fc6-b794dcf1fcab.png)

### Limitations:
- You can only assert on parameters, variables, and resource names with current functionality
- You cannot assert on resource size, location, or other propreties yet as these depend on runtime deployment functionality

Contributor guide

Open the contributing guide

Research direction

Read the linked proposal in issue #11966 first. The issue identifies bicepconfig.json and the `bicep test ` CLI entry point, but names no implementation files or tests. A concrete change and completion criteria must be established before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cli, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.