manticoresoftware / manticoresoftware/doc-tests-playwright
Playwright test framework for Manticore docs
Open
@PavelShilin89 is already working on this.
Since Mar 5, 2026.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Context
Created a new test framework for manual.manticoresearch.com as a replacement for the existing doc-tests (Python + Selenium).
Why migrate from Selenium
| Selenium (old) | Playwright (new) | |
|---|---|---|
| Speed | ~5 min (18 time.sleep() calls) |
~50s (zero sleeps) |
| Tests | 18 | 34 |
| Retries | None | 2 automatic on CI |
| Video on failure | No | Yes |
| Trace (step-by-step replay) | No | Yes |
| HTML report | No | Yes |
| Artifact retention (CI) | 1 day | 14 days |
| Infrastructure | Selenium Grid (2 containers) | 1 Playwright image |
| Language | Python | TypeScript |
What's done
Core framework (branch: main, merged)
- 19 tests migrated 1:1 from Selenium
playwright.config.tswith baseURL, retries, video/trace/screenshot on failure- Page Objects:
ManualPage,CodeTabsPage - Docker environment (Dockerfile + docker-compose.yml)
- GitHub Actions CI (daily + PR triggers)
Extended coverage (branch: feat/extended-coverage, PR #1)
- 15 additional tests (34 total)
- navigation, anchors, copy-button, broken-links, page-404, mobile-viewport
AI Test Assistant (branch: feat/ai-test-assistant, PR #2)
- Web dashboard at localhost:3000 with chat UI powered by Claude API
- AI can generate new tests, run them, analyze failures, create PRs
- 7 tools: list_tests, read_file, write_file, run_tests, git_status, create_branch, create_pr
- Structured test results panel with pass/fail breakdown
- Dark/light theme
Smart Reporter integration (branch: feat/ai-test-assistant)
- Integrated playwright-smart-reporter
- Stability grades (A-F), flakiness detection, step timeline, trend analytics
- Free tier — no license needed for core features
Test coverage
| File | What it tests | Tests |
|---|---|---|
basic-connection.spec.ts |
Site loads | 1 |
search.spec.ts |
Search works, results clickable | 2 |
code-tabs.spec.ts |
SQL/HTTP/PHP/Python tab switching | 6 |
language-switcher.spec.ts |
EN/RU/ZH switching and persistence | 4 |
os-tabs.spec.ts |
RHEL/Debian/Docker/MacOS/K8s tabs | 6 |
navigation.spec.ts |
Sidebar sections and links | 4 |
anchors.spec.ts |
Deep linking with anchors | 2 |
copy-button.spec.ts |
Code copy button | 2 |
broken-links.spec.ts |
Link integrity (HTTP 200) | 2 |
page-404.spec.ts |
404 error handling | 2 |
mobile-viewport.spec.ts |
Mobile responsiveness (iPhone 12) | 3 |
Roadmap / ideas
- Merge PR #1 (extended coverage) and PR #2 (AI assistant + smart reporter)
- Add CI history caching for flakiness detection across runs
- Explore Playwright CLI for AI agents for token-efficient browser automation
- Consider StageWright Starter license for AI failure analysis in reports
- Add tests for more documentation pages (Installation, Server settings, etc.)
- Add accessibility tests
- Add visual regression tests (screenshot comparison)
- Explore integration with existing CI/CD pipelines
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.