microsoft / microsoft/AI-Engineering-Coach

Weak assertion in timeline e2e test always passes regardless of nav controls

Open
#99 0 comments 0 reactions 1 assignee View on GitHub

@mc5eamus is already working on this.

Since Jun 7, 2026.

bug
Dominant language
TypeScript
Stars
4.2k
Forks
585
Avg merge
22h 5m
Merged PRs (30d)
16

Description

Summary

The navigation controls visible assertion in tests/e2e/timeline.spec.ts is effectively a no-op — it passes even when the navigation controls are missing.

test(''navigation controls visible'', async ({ page }) => {
  // Should have prev/next day navigation
  const content = await page.textContent(''#content'');
  expect(content).toMatch(/May|2026|prev|next|←|→/i);
});

Why it''s broken

The regex is an OR of six alternatives, and the timeline view always renders text that satisfies at least one branch regardless of navigation state:

  1. The CSS class session-preview (rendered for every session row in src/webview/page-timeline.ts around line 246) contains the substring prev, which matches /prev/i against textContent. Even if no nav buttons rendered, the assertion would pass as long as any session row exists.
  2. The May / 2026 branches are fixture-bound (they come from tl.date formatted via toLocaleDateString in page-timeline.ts:98), so regenerating the fixture to a different month/year silently kills those branches without anyone noticing — because the prev branch already always matches.
  3. Net effect: the test does not actually verify that the prev/next navigation controls are visible. A regression that removes the nav buttons would not fail this test.

Related

Surfaced while reviewing #88, which fixed a similar (but stronger) time-bomb assertion in burndown.spec.ts. This one is weaker but more insidious because it silently always passes rather than failing on a known date.

Suggested fix

Assert the navigation controls directly instead of pattern-matching free text, e.g.:

test(''navigation controls visible'', async ({ page }) => {
  await expect(page.locator(''button:has-text("Prev")'')).toBeVisible();
  await expect(page.locator(''button:has-text("Next")'')).toBeVisible();
});

Or target whatever stable selector / data-testid the prev/next controls expose in the timeline page.

Acceptance criteria

  • tests/e2e/timeline.spec.ts navigation controls visible asserts on the actual prev/next nav controls (locator/selector based), not a free-text regex.
  • Removing or hiding the prev/next nav buttons in src/webview/page-timeline.ts causes the test to fail.
  • No hard-coded month/year strings remain in the assertion.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.