kubeflow / kubeflow/dashboard

(feat): Consolidate E2E Testing: Migrate from Puppeteer + Cypress to Playwright

Open
#227 8 comments 0 reactions 1 assignee Claimed by @naman9271 View on GitHub
Dominant language
TypeScript
Stars
19
Forks
68
Avg merge
2d 16h
Merged PRs (30d)
10

Description

## Summary

The repo has **two separate, incompatible E2E setups** that share no tooling, no reporting, and no runner. This issue proposes migrating both to **Playwright** as the single canonical E2E framework.

---

## Current State

### 1. `components/centraldashboard`: Puppeteer v1.20 + Jasmine v3

- **Puppeteer `^1.20.0`** (Oct 2019, bundles Chromium 77 hundreds of unpatched CVEs)
- **Jasmine `^3.5`** as the test runner via `ts-node`; `@types/puppeteer ^2.0.0` mismatches the actual version
- Requires a live GKE cluster + 4 cloud credential env vars **cannot run in standard CI**
- Uses `page.waitForFunction().asElement().click()` an API removed in Puppeteer v5+

**What is tested:**
- First-time user registration flow
- Shadow DOM navigation (`main-page` → `registration-page`)
- Namespace selector after registration
- GCloud OAuth token flow

**What is NOT tested:**
- Returning / existing user login flow
- Multi-namespace switching from the UI
- Dashboard sidebar navigation to any sub-application
- Error states (profile creation failure, network errors, API 500s)
- Anything without a live GKE cluster there is **no mocking layer**
- Cross-browser (only the locally installed Chrome binary is used)
- Mobile / responsive layouts

---

### 2. `components/centraldashboard-angular/frontend` Cypress v13.13.1

- **Cypress `~13.13.1`** with Mocha runner; 2 spec files, 10 custom commands
- Requires `cy.on('window:before:load', win => { win.parent = win; })` in every `beforeEach` because Cypress itself runs inside an iframe a structural hack
- `modifyObstructiveCode: false` required in config for the same reason
- No WebKit/Safari support; parallel execution needs paid Cypress Cloud

**What is tested:**
- Namespace selection priority (URL param > localStorage > first owner NS > first NS)
- Invalid namespace rejection; "All namespaces" path allowlist
- Browser URL ↔ iframe URL sync during navigation and cross-WA link clicks

**What is NOT tested:**
- Authentication / login flow all tests assume a pre-authenticated session
- Profile or namespace creation through the UI
- Sidebar collapse / responsive behaviour
- Cross-origin iframes Cypress has an architectural limit here; sub-apps served from different origins are untestable
- Error / empty states (no namespaces, API failures)
- User role differences (Owner vs Contributor) in the UI
- Keyboard navigation / accessibility
- WebKit (Safari) only Chromium and Electron are used in CI
- Notification / banner component behaviour

---

## Why Two Stacks Is a Problem

The project runs **three different test runners** (Jasmine, Cypress/Mocha, Karma/Jasmine for unit tests) with no shared reporting, no unified CI step, and a Puppeteer stack that is a security liability requiring live cloud infrastructure.

---

## Proposed Solution: Playwright

[Playwright](https://playwright.dev/) (Microsoft) replaces both Puppeteer and Cypress with a single `@playwright/test` package supporting Chromium, Firefox, and WebKit.

### Why Playwright wins

| | Puppeteer v1 + Jasmine | Cypress v13 | **Playwright** |
|---|---|---|---|
| Iframe support | Manual shadow DOM | `contentDocument` hack | `page.frameLocator()` first class |
| Browsers | Chrome only | Chrome, Firefox (no WebKit) | Chromium, Firefox, **WebKit** |
| Network mocking | None | `cy.intercept()` | `page.route()` |
| Parallel execution | No | Paid (Cypress Cloud) | Built-in, free |
| Test runner | Jasmine | Mocha | `@playwright/test` (replaces both) |
| HTML reports | No | Plugin needed | Built-in |
| Security | Chromium 77 CVEs | N/A | Ships up-to-date Chromium |
| TypeScript types | Mismatched | `declare global` hacks | Ships own types |
| Maintenance | Abandoned (v1) | Active | Active (weekly releases) |

### API translation (quick reference)

```ts
// Cypress → Playwright
cy.get('iframe').its('0.contentDocument.body').then(b => cy.wrap(b)).find('button').click();
// → await page.frameLocator('iframe').getByRole('button').click();

cy.intercept('GET', '/api/dashboard-links', { fixture: 'dashboardlinks' });
// → await page.route('/api/dashboard-links', r => r.fulfill({ path: 'fixtures/dashboardlinks.json' }));

cy.get('[data-cy-selected-namespace]').should('contain', 'my-ns');
// → await expect(page.locator('[data-cy-selected-namespace]')).toContainText('my-ns');
```

---

## Migration Scope

### Phase 1 `centraldashboard-angular` (Cypress → Playwright)
- Replace `cypress.config.ts` → `playwright.config.ts`
- Migrate 2 spec files; replace `commands.ts` with Page Object helpers
- Remove `cypress`; add `@playwright/test ^1.44`
- Update `ui-test` / `ui-test-ci` / `ui-test-ci-all` scripts

### Phase 2 `centraldashboard` (Puppeteer + Jasmine → Playwright)
- Rewrite `test/e2e.test.ts` using `@playwright/test`
- Replace `JWT.authorize()` with Playwright `request` fixture + `storageState`
- Remove `puppeteer`, `@types/puppeteer`, `jasmine`, `jasmine-console-reporter`; add `@playwright/test ^1.44`

---

## Acceptance Criteria

- [ ] All existing Cypress + Puppeteer test scenarios pass under Playwright
- [ ] `cypress` and `puppeteer` removed from both `package.json` files
- [ ] Single `@playwright/test` version across the monorepo
- [ ] CI steps in `testing/shared/` and `testing/gh-actions/` updated to run `playwright test`
- [ ] HTML report artifact uploaded in GitHub Actions
- [ ] Component `README.md` files updated

## Estimated Effort

| Phase | Effort |
|---|---|
| Phase 1 Cypress → Playwright | 2–3 days |
| Phase 2 Puppeteer → Playwright | 1 day |
| CI / build updates | 0.5 day |
| **Total** | **~3.5–4.5 days** |

---

## References

- [Playwright docs](https://playwright.dev/docs/intro)
- [Cypress → Playwright migration guide](https://playwright.dev/docs/migration)
- [Puppeteer v1.20 CVEs (Snyk)](https://security.snyk.io/package/npm/puppeteer/1.20.0)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.