WordPress / WordPress/php-toolkit
E2E tests for the entire Blueprint execution flow
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 61
- Forks
- 21
- Avg merge
- 19h 34m
- Merged PRs (30d)
- 7
Description
TL;DR I am drawn to a conclusion that e2e testing for all input types will make test development and runs longer - with little impact on quality. I'd rather focus on better unit testing, as per observations, below.
With a lot of test development done in #82 it is time to garther some thoughts on how to meaningully test this lib.
First of all: #82 has been a constant source of new issues, like #104 #107 or #83 which means e2e really do impact the quality of our work here. It's a trivial conclusion but still - nice to confirm.
Next: I've started with a pipeline running parameterized tests for all types of inputs. This was quickly reduced by one, when we decided to drop BlueprintBuilder as an input type with #98 With this done, I took a moment to examine the main process flow and see how it could be tested best.
- The process is initiated with the
run_blueprintfunction; the DI, runtine and env are set up first. - The input is preprocessed with
parseAndCompile:- Parsing depends on the type of the input:
- for a JSON string it is decoded and validated. and mapped to a
Blueprint, - for a stdClass it is validated and also mapped to a
Blueprint, - and a
Blueprintis just validated.
- for a JSON string it is decoded and validated. and mapped to a
- Compiling always takes a
Blueprintand returns aCompiledBlueprint. At this point all paths have allready converged.
- Parsing depends on the type of the input:
- The
CompiledBlueprintis run by theBlueprintRunner. Each run produces a result which is an array ofStepSuccessorBlueprintRunnerException. At this point it time, the return from the runner is not transformed in any way or form by the engine or the most externalrun_blueprintfunction.
Observations:
- Both validation and mapping are identical for a JSON string and stdClass.
- Decoding is done trivially with
json_decodeso there is nothing to trully test. - Mapping is done via the
BlueprintMapperI wrote some time ago. I'd gladly test it more thoroughly but this can be done via unit tests. - Validation is done identically for all types, with a small caveat for the
Blueprinttype. This is a work-around so it is likely to be rewritten. But either way it can be completely tested with unit tests. - For top notch quality
parseAndCompilecould be tested as a whole subprocess. Those test will be quick, so there is little worry for test execution times if they are run for all input types. - What actually makes E2E tests run long is the downloading and installing of resources. So doing it trice mostly triples the pipeline execution flow, with no return on investment.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names run_blueprint, parseAndCompile, BlueprintMapper, and BlueprintRunner. Read those entry points and the existing tests first; determine which subprocess or integration coverage is intended for the converged flow and input validation. Done would require an explicit test scope and passing coverage, but the issue does not currently define that scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100