opensearch-project / opensearch-project/data-prepper

[BUG] Smoke tests fail in GitHub Actions

Open
#2,579 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug

The release build's smoke tests fail consistently.

However, I can run them locally with success.

To Reproduce
Steps to reproduce the behavior:

  1. Run a release build GHA

Expected behavior

The tests pass (perhaps with a re-run failed jobs)

Sample run

https://github.com/opensearch-project/data-prepper/actions/runs/4758053910/jobs/8455993252

Ready to begin smoke tests. Running cURL commands.

Test: Verify logs received via HTTP were processed
Open Search is receiving logs from Data Prepper
Found at least 78 hits
Test passed
Open Search successfully received logs from Data Prepper!

Test: Verify metrics received via grpc were processed
No hits found with query url https://localhost:9200/otel-v1-apm-span-000001/_search?q=PythonService
Smoke test failed
Stopping smoke-tests_otel-span-exporter_1  ... 
Stopping smoke-tests_otel-collector_1      ... 
Stopping smoke-tests_data-prepper_1        ... 
Stopping node-0.example.com                ... 
Stopping smoke-tests_http-log-generation_1 ... 
Stopping smoke-tests_http-log-generation_1 ... done
Stopping smoke-tests_otel-span-exporter_1  ... done
Stopping smoke-tests_otel-collector_1      ... done
Stopping smoke-tests_data-prepper_1        ... done
Stopping node-0.example.com                ... done
Removing smoke-tests_otel-span-exporter_1  ... 
Removing smoke-tests_otel-collector_1      ... 
Removing smoke-tests_data-prepper_1        ... 
Removing node-0.example.com                ... 
Removing smoke-tests_http-log-generation_1 ... 
Removing node-0.example.com                ... done
Removing smoke-tests_otel-collector_1      ... done
Removing smoke-tests_otel-span-exporter_1  ... done
Removing smoke-tests_http-log-generation_1 ... done
Removing smoke-tests_data-prepper_1        ... done
Removing network smoke-tests_default
Smoke tests failed
Error: Process completed with exit code 1.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the release build GitHub Actions workflow and the smoke-test entry point that runs the cURL checks. Reproduce the metrics test failure from the linked sample run, then compare its GitHub Actions environment with a successful local run. Done means the release smoke tests reliably find the processed metrics in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, java
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.