elastic / elastic/apm-agent-python
[CI] BDD Tests that are not implemented: should the build fail or warn?
- Dominant language
- Python
- Stars
- 431
- Forks
- 239
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 7
Description
We noticed that there are a few BDD files under https://github.com/elastic/apm-agent-python/tree/master/tests/bdd/features
- api_key
- azure_app_service_metadata
- outcome
But only the API Key ones are implemented.
>[2021-08-02T10:01:23.365Z] tests/bdd/__init__.py 0 0 0 0 100%
[2021-08-02T10:01:23.365Z] tests/bdd/api_key_tests.py 31 15 0 0 52%
[2021-08-02T10:01:23.365Z] tests/client/__init__.py 0 0 0 0 100%
You can check that they do not appear under the test list for that package: https://apm-ci.elastic.co/job/apm-agent-python/job/apm-agent-python-mbp/job/master/500/testReport/tests.bdd/
I'd like to ask about your expectations on it: should the build fail because they are not implemented, or show them as skipped?
Contributor guide
Research direction
Start by reviewing the BDD files under tests/bdd/features for api_key, azure_app_service_metadata, and outcome, then compare them with tests/bdd/api_key_tests.py and the linked CI test report. The issue needs an agreed policy for unimplemented scenarios before implementation can be scoped; done means the chosen fail-or-skip behavior is explicit and consistently reflected in CI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100