apache / apache/streampark

[Console] Migrate E2E (Selenium) to API integration tests

Open
#4,477 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
4.3k
Forks
1.1k
Avg merge
13h 35m
Merged PRs (30d)
2

Description

## Background

Current E2E (`streampark-e2e`) depends on:

- Full Docker image build (`build.sh` + `docker/Dockerfile`)
- Testcontainers + docker-compose
- Selenium Chrome (UI automation against legacy webapp `streampark-console-webapp`)

This causes:

1. **High flakiness** — Selenium timeouts, element not interactable, CI `cancel-in-progress` cascading cancellations
2. **Long CI runtime** — E2E-Build ~7 min + matrix jobs ~5–7 min each (often 30–60 min total)
3. **Tight coupling to UI/DTO wire format** — backend API behavior can be correct while E2E fails due to wire format or `@Valid` strictness (e.g. Alarm list JSON string vs object, `resourcePath` null copy)
4. **High maintenance** — page objects break on Ant Design / i18n / layout changes

**Goal:** Replace UI-driven E2E with **API integration tests** in `streampark-console-service`, covering the same business flows via REST without a browser or frontend.

## Scope

**In scope (migrate from E2E matrix):**

| Current E2E class | API integration test target |
|-------------------|----------------------------|
| `EnvironmentTest` | Flink/Spark env CRUD + default |
| `AlarmTest` | `/flink/alert/*` CRUD + exists + send |
| `UserManagementTest` | `/user/*` CRUD + password |
| `RoleManagementTest` | `/role/*` |
| `TeamManagementTest` | `/team/*` |
| `MemberManagementTest` | `/member/*` |
| `ExternalLinkTest` | `/flink/externalLink/*` |
| `YarnQueueTest` | `/yarn/queue/*` + check |
| `TokenManagementTest` | `/token/*` |
| `UploadManagementTest` | `/resource/*` + upload |
| `ProjectsManagementTest` | `/project/*` |
| `VariableManagementTest` | `/variable/*` |

**Out of scope / separate track:**

- `Flink120OnRemoteClusterDeployTest` — real Flink cluster deploy; keep as optional heavy E2E or dedicated deploy test job

**Not changing (this issue):**

- Legacy webapp (`streampark-console-webapp`)
- webapp-v2

## Proposed architecture

```
streampark-console/streampark-console-service/src/test/java/
org/apache/streampark/console/
integration/ # NEW package
base/
ApiIntegrationTestBase.java # @SpringBootTest(RANDOM_PORT) + auth helper
ApiClient.java # TestRestTemplate wrapper, JWT/Shiro session
cases/
AlarmApiIT.java
UserApiIT.java
...
```

**Stack:**

- `@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)`
- `TestRestTemplate` or `@AutoConfigureMockMvc` + `MockMvc`
- Reuse H2 in-memory DB + existing test SQL (`SpringUnitTestBase` patterns)
- Shiro/JWT: login via `POST /passport/signin`, reuse token for `@Permission` endpoints
- Assert `RestResponseBody` envelope: `code`, `data`, `message`, legacy `extra` fields

**Request paths to cover explicitly:**

- `@FormOrJson` form-urlencoded path (legacy webapp default)
- `@RequestBody` JSON path (Alert, OpenAPI)
- Query/form params for list/page endpoints

## Implementation phases

### Phase 0 — Infrastructure (1 PR)

- [ ] `ApiIntegrationTestBase` with admin login, team context, helper methods
- [ ] `ApiClient.postForm()`, `postJson()`, `deleteForm()`, multipart upload
- [ ] Sample `PassportApiIT` or `HealthApiIT` as template
- [ ] Maven: ensure integration tests run in `./mvnw test -pl streampark-console-service`
- [ ] Document conventions in test class Javadoc

### Phase 1 — Settings module (1 PR)

Migrate: Alarm, YarnQueue, ExternalLink, Variable, Environment (Flink/Spark env)

- Wire-format regression tests (e.g. alert list returns JSON **strings** for `*Params`)
- Check/exists endpoints use slim DTOs, not full create DTO + `@Valid`

### Phase 2 — System module (1 PR)

Migrate: User, Role, Team, Member, Token

- RBAC: test with admin vs non-admin where applicable
- Token create legacy `extra("code", 0)` behavior

### Phase 3 — Resource & Project (1 PR)

Migrate: Upload/Resource, Projects

- Multipart upload via MockMvc
- Project build/build_log polling (assert `offset` / `readFinished` extras)

### Phase 4 — CI & deprecation (1 PR)

- [ ] New workflow `.github/workflows/integration-test.yml`:
- JDK 11, `./mvnw test -pl streampark-console/streampark-console-service`
- No Docker image build, no Selenium
- Target: < 10 min total
- [ ] Mark E2E matrix jobs as `continue-on-error` or move to nightly
- [ ] Update `AGENTS.md` testing section
- [ ] Eventually remove or archive `streampark-e2e` (separate issue/PR after soak period)

## Test design principles

1. **Test behavior, not UI** — assert HTTP status + `RestResponseBody` + DB state
2. **One endpoint contract per assertion** — avoid mega-tests
3. **Legacy wire compatibility** — form field names match webapp; document in test names
4. **No `@Valid` on progressive check endpoints** — match service-layer status codes
5. **Idempotent setup** — `@Transactional` or `@Sql` per class; unique names with UUID suffix
6. **Do not duplicate unit tests** — integration = full stack Controller → Service → DB

## Unit vs integration vs E2E boundary

| Layer | What to test | Example |
|-------|--------------|---------|
| Unit | Service/Assembler logic | `UserServiceTest` |
| Integration | Controller → DB full path + HTTP envelope | `AlarmApiIT` |
| E2E (keep few) | Real Flink/K8s deploy | `Flink120OnRemoteClusterDeployTest` |

## Acceptance criteria

- [ ] All 12 E2E cases above have API IT equivalents with ≥ same scenario coverage
- [ ] CI integration-test job green on PR to `dev`
- [ ] E2E workflow no longer blocks PR merge (nightly or removed)
- [ ] No dependency on Selenium/Testcontainers for console CRUD flows
- [ ] Documented migration mapping E2E class → IT class in issue or `docs/`

## Risks & mitigations

| Risk | Mitigation |
|------|------------|
| Shiro permission hard to simulate | Base class admin login; `@MockBean` only for PermissionAspect unit tests |
| FormOrJson dual path missed | At least 1 form + 1 json case per write endpoint (Alert etc.) |
| H2 dialect differences | Reuse existing test SQL; MySQL-specific syntax via unit + manual validation |
| Team used to Selenium debug | Log HTTP request/response body on IT failure |

## References

- Current E2E workflow: `.github/workflows/e2e.yml`
- DTO migration PR: #4475
- Recent E2E failures: DTO wire format, `@Valid` strictness, Selenium flakiness

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with .github/workflows/e2e.yml, the existing streampark-e2e classes, and SpringUnitTestBase patterns in streampark-console-service. Begin with the Phase 0 integration-test base and client, then use the listed E2E-to-API mapping to plan the remaining cases. Done means all 12 cases have API integration equivalents, the integration-test workflow is green, and Selenium no longer blocks console CRUD changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, java, spring-boot
Domain
backend-api-design, ci-cd, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.