ci(workflow): Discuss selective execution of acceptance tests based on changed paths
- Dominant language
- JavaScript
- Stars
- 1.1k
- Forks
- 397
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 25
Description
## Description
While reviewing the `pr-testing-with-test-project.yml` workflow, I noticed that the acceptance tests currently run for all three language templates (`js`, `py`, and `java`) whenever `should_test` is `true`, regardless of which part of the repository was modified.
After analyzing the workflow, test infrastructure, and template dependencies, I think there may be an opportunity to reduce unnecessary acceptance test executions. Before working on an implementation, I'd like to discuss whether this approach aligns with the project's expectations.
### Current behavior
The acceptance workflow uses a matrix:
```yaml
strategy:
matrix:
language: [js, py, java]
```
Once the `changes` job sets `should_test=true`, all three acceptance suites are executed.
---
### Findings from the repository analysis
#### 1. Language-specific acceptance tests are already isolated
The acceptance workflow executes one Docker Compose profile per matrix job:
* `test-js`
* `test-py`
* `test-java`
Each profile starts only its corresponding tester service.
#### 2. Shared infrastructure is independent
The workflow starts the common Microcks infrastructure using the `ci` profile before executing the language-specific profile.
The language-specific tester services do not depend on one another, only on the shared infrastructure.
#### 3. Repository has both shared and language-specific areas
From tracing the acceptance test dependency chain, there appear to be two categories of paths:
**Shared (should trigger all acceptance tests):**
* `apps/generator/**`
* `packages/helpers/**`
* `packages/components/**`
* `apps/react-sdk/**`
* Shared acceptance infrastructure (`integration-test`, `__fixtures__`, `microcks-setup`, etc.)
**Language-specific (could potentially trigger only one acceptance suite):**
* `packages/templates/clients/websocket/javascript/**`
* `packages/templates/clients/websocket/python/**`
* `packages/templates/clients/websocket/java/quarkus/**`
---
### Discussion
Would it make sense to extend the existing `paths-filter` configuration so that:
* changes to shared packages continue running all acceptance tests,
* while changes limited to a single client template execute only that client's acceptance suite?
The main goal would be to reduce unnecessary CI execution time without skipping any required validation.
---
Contributor guide
Research direction
Start by reading pr-testing-with-test-project.yml, especially the changes job, paths-filter configuration, and language matrix. Review the ci, test-js, test-py, and test-java Docker Compose profiles and confirm the dependency paths listed in the issue. Done means an agreed design that runs only affected acceptance suites while preserving all-suite runs for shared changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, github-actions
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100