elastic / elastic/elastic-package
Add automated tests with dashboards
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
Currently the only tests we have with dashboards check that they can be installed during assets tests. Having functional tests would allow us to check that the dashboards actually work, and to have additional checks on the data collected.
We need to find a way to check that the data collected is relevant for dashboards, and that dashboards have enough data to render correctly.
Some ideas to explore:
* Extract queries from dashboard definitions. Some panels contain queries that could be executed from a test script.
* Look for programatic ways to check if the dashboard has any panel with errors.

* The inspect view in Kibana dashboards is able to show the queries executed by the panel, if there is some API to access this information we could run the queries during system tests and check that the collected data is valid and relevant for dashboards.


* Add a dashboards test runner (`elastic-package test dashboards`) that uses data generation like the one provided by `elastic-package benchmark stream` to test the dashboards.
* Deploy some MITM proxy between Kibana and Elasticsearch and capture queries done when opening dashboards.
* Use webdriver or similar to do headless browser testing.
* Screenshot-based tests could help to validate that the actual rendered dashboard makes sense, but we would need to be able to control all the data represented in a deterministic way.
The implementation could be incremental, adding features in different areas of testing.
cc @lalit-satapathy
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing assets tests and dashboard definitions, then examine the `elastic-package benchmark stream` data-generation entry point. Compare the proposed query extraction, dashboard error checks, proxy, webdriver, and screenshot approaches. Done means an incremental dashboard test capability can verify collected data and that dashboards have enough valid data to render.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, go
- Domain
- cli, data-visualization, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100