elastic / elastic/apm-agent-python

[CI] BDD Tests that are not implemented: should the build fail or warn?

Open
#1,201 2 comments 0 reactions 0 assignees View on GitHub
agent-python question
Dominant language
Python
Stars
431
Forks
239
Avg merge
5d 10h
Merged PRs (30d)
7

Description

We noticed that there are a few BDD files under https://github.com/elastic/apm-agent-python/tree/master/tests/bdd/features
- api_key
- azure_app_service_metadata
- outcome

But only the API Key ones are implemented.

>[2021-08-02T10:01:23.365Z] tests/bdd/__init__.py 0 0 0 0 100%
[2021-08-02T10:01:23.365Z] tests/bdd/api_key_tests.py 31 15 0 0 52%
[2021-08-02T10:01:23.365Z] tests/client/__init__.py 0 0 0 0 100%

You can check that they do not appear under the test list for that package: https://apm-ci.elastic.co/job/apm-agent-python/job/apm-agent-python-mbp/job/master/500/testReport/tests.bdd/

I'd like to ask about your expectations on it: should the build fail because they are not implemented, or show them as skipped?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the BDD files under tests/bdd/features for api_key, azure_app_service_metadata, and outcome, then compare them with tests/bdd/api_key_tests.py and the linked CI test report. The issue needs an agreed policy for unimplemented scenarios before implementation can be scoped; done means the chosen fail-or-skip behavior is explicit and consistently reflected in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.