elastic / elastic/logstash

[meta] The workflow of testing Logstash

Open
#5,139 4 comments 1 reaction 0 assignees View on GitHub
discuss meta tests-infra
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

This is a proposal of definition for a testing workflow for the project and it's plugins,
### Logstash Core
- `rake test:core`
- Run every commit.
- with Released logstash-core
- with the Local code.
- `rake test:dependencies`
- Makes sure logstash start with default plugins.
- Run daily.
- `rake test:acceptance` #5070
- Runs after test:core is green.
- Runs every commit.
- `rake test:integration`
- Runs daily
- `rake test:plugin` (testing a plugin with master)
- Help for devs to know current status
- Run daily
- `rake test:smoke`
- Runs quick validation for release QA
- Hands on Workshop config.
- Runs on demands (daily)
- PR test on travis.
### Logstash plugins
- Testing plugin with last released version.
- Run per commit.
- PR testing on travis.
#### Test description:
- `rake test:core` : It makes sure logstash core functionality is properly implemented, might include unit and some pipeline integration test (for filters)
- `rake test:dependencies` : it makes sure there are no unexpected dependency conflicts between plugins.
- `rake test:acceptance` : it makes sure the RC artifacts are in good shape for being in release quality.
- `rake test:integration` : it makes sure LS works as expected with 3 party components like Elasticsearch, RabbitMQ, Redis, etc.
- `rake test:plugin` : It makes sure a plugin ( still not released ) works as properly with current LS core code.
- `rake test:smoke` : Runs a subset of the full integration and acceptance test to green light artifacts.

Detailed issues will be created for each one of this points.

related #5013

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the proposed rake tasks in this issue and the related issue #5013. Compare the listed core, dependency, acceptance, integration, plugin, smoke, and Travis workflows with the current project setup. This issue is complete when the testing workflow is agreed and the promised detailed issues exist for each point.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.