elastic / elastic/integrations

[CI] Migrate dev scripts test suite to bats framework

Open
#19,153 2 comments 0 reactions 0 assignees View on GitHub
Team:Ecosystem
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

## Context

PR #19104 introduced `.buildkite/scripts/run_dev_scripts_tests.sh`, a hand-rolled Bash test runner for scripts in `dev/scripts/`. It works but carries a TODO to migrate to the [bats-core](https://github.com/bats-core/bats-core) framework.

## Problem

The current test runner is a custom Bash script with manual `pass`/`fail` counters, no test isolation, and no structured output. This makes it harder to:

- Diagnose failures in CI (no TAP/JUnit output format)
- Add new tests safely (no native `setup`/`teardown` hooks per test)
- Maintain test independence (shared state leaks between test cases)

## Proposed work

Migrate `.buildkite/scripts/run_dev_scripts_tests.sh` to the [bats-core](https://github.com/bats-core/bats-core) framework:

- Replace custom `assert_equals` / `assert_exit_code` helpers with `bats` assertions
- Use `setup` / `teardown` for dummy repo lifecycle instead of the `trap cleanup EXIT` pattern
- Emit TAP output (native in bats) so Buildkite can parse test results
- Add bats as a CI dependency (or use the Docker image already available in the pipeline)

## Broader opportunity

Adopting bats would also give us a foundation to start testing CI scripts in `.buildkite/scripts/` that are currently untested. Many of those scripts contain non-trivial logic (pipeline branching, artifact handling, environment checks) but have no test coverage today. Having bats already wired into the pipeline would lower the barrier to adding tests for them incrementally.

## Acceptance criteria

- [ ] Tests for `get_release_commit.sh` pass under bats with the same coverage as today
- [ ] TAP/JUnit output is available to Buildkite
- [ ] `setup`/`teardown` replace the current `DUMMY_REPO` + trap pattern
- [ ] Any future scripts added to `dev/scripts/` or `.buildkite/scripts/` can be tested by adding a `.bats` file

## Related

- PR #19104 (introduced the current test suite)

---

*Generated with [Claude Code](https://claude.ai/claude-code)*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.