felladrin / felladrin/MiniSearch

test: expand E2E test coverage beyond smoke tests

Closed
#2,517 0 comments 0 reactions 0 assignees View on GitHub
test
Dominant language
TypeScript
Stars
587
Forks
70
Avg merge
1h 42m
Merged PRs (30d)
184

Description

## Problem

The E2E test suite (`e2e/`) contains only 2 smoke tests:

1. Homepage loads, search runs, and results render
2. A failing text search degrades to the retryable alert

These tests verify basic functionality but leave major features untested:

- **AI response generation** (in-browser and remote)
- **Search history** (save, restore, clear)
- **Settings persistence** (inference type, model selection)
- **Access key flow** (validation, timeout)
- **Image search**
- **Conversation memory** (follow-up questions)
- **Analytics** (search stats, activity heatmap)

## Why It Matters

- Smoke tests catch deployment failures but not feature regressions
- The AI response flow is the project's differentiator, but it's not tested end-to-end
- Settings and history bugs are user-facing and frustrating
- The E2E infrastructure (Playwright, Docker container check) is already in place

## Current E2E Setup

- **Framework**: Playwright (Chromium only)
- **Runner**: `npm run test:e2e`
- **CI**: Runs in `reusable-check-docker.yml` against a production Docker container
- **Timeout**: 90 seconds per test
- **Retries**: 1 on CI

## Proposed Test Targets

1. **AI Response Flow**
- Enable AI response in settings
- Perform a search with AI enabled
- Verify response renders with citations
- Verify follow-up question works

2. **Search History**
- Perform multiple searches
- Open history drawer
- Verify searches are listed
- Restore a previous search
- Clear history

3. **Settings Persistence**
- Change inference type
- Reload page
- Verify setting persists

4. **Access Key Flow**
- Set access key in environment
- Verify access page appears
- Enter correct key
- Verify access granted

## Acceptance Criteria

- E2E tests added for at least 3 of the 4 priority areas above
- Tests run against the production Docker container (same as CI)
- `npm run test:e2e` passes
- CI E2E step continues to pass
- Tests are resilient to timing issues (use Playwright's auto-waiting)

Contributor guide

Open the contributing guide

Research direction

Read the existing tests in e2e/ and run npm run test:e2e to understand the current Playwright setup. Add coverage for at least three listed priority areas, using Playwright auto-waiting, then verify the tests pass against the production Docker container and that the E2E step in reusable-check-docker.yml continues to pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, playwright, typescript
Domain
ci-cd, testing, web-dev
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.