MithrilJS / MithrilJS/mithril.js

Our tests should be running in real browsers as well as Node

Open
#2,359 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Area: Workflow Type: Bug Type: Meta/Feedback
Dominant language
JavaScript
Stars
14.5k
Forks
922
PR merge metrics
No merged PRs in 30d

Description

We need to start testing our code in real browsers and in ES5 environments to avoid issues like this, reported on Gitter. It'd also make it much easier to detect and test for IE bugs, if we could get on either BrowserStack's or Sauce Labs' open source plans, but we first need to get our end together so we can even use it.

Here's the state of everything:

  • All our browser tests have a web page with all the relevant tests loaded via <script src="..."></script>.
  • Our browser test pages rely on a require pseudo-polyfill that doesn't even wrap modules in a closure when executing them.
  • Our tests exclusively use DOM mocks, missing 100% of browser-specific bugs.
  • We're only actively testing it in the latest Node, using our own mocks instead of something like JSDOM.

This is low priority for now, but I am working on it partially in my local Rollup migration branch (slated for a future PR). Here's my plan, some of which I've already got resolved locally:

  • Migrate the tests to grab their mocks from a global instance, created from a test bundle rather than the individual files.
  • Migrate the web pages to load only the tests and the generated test bundle.
  • Make ospec on npm test load the generated test bundle before executing tests in Node.
  • Create a karma-ospec to tell Karma how to handle ospec results
    • I'll need to add a feature for async test reporting, so Karma doesn't go ballistic over not receiving a single event in the ~3 seconds it takes to run everything, in case the browser briefly disconnects in the middle of test execution. (Karma's timeout is 2000ms = 2 seconds, which is a problem.)
  • Switch our browser test runner to Karma
  • Set up Travis to run it in Chrome, Firefox, and Safari using their respective launchers.
    • This is probably going to be the easiest part of this.
  • Migrate the DOM tests off of the mocks as much as practically possible.
    • Testing for ev.preventDefault() is as simple as checking ev.defaultPrevented
    • Testing for ev.stopPropagation() is as simple as firing an event that can propagate and ensuring it doesn't get found by the parent.
    • The valueSetter and typeSetter spies could just be replaced with an interim hack for elem.value and input.type, which we'd remove once we incorporate IE into the test matrix. (We can do this on Travis with their early stage Windows CI support, but note that Edge is not available without something like Sauce Labs or BrowserStack.)
  • Set up Travis to run tests in IE via karma-ie-launcher. This includes the promise polyfill, streams, and everything.
  • And then, we can then confidently say things actually work and aren't a house of cards ready to fall over and collapse.

As for the above, I only locally have the first done completely and the second done for /api/tests/*.js only.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with /api/tests/*.js, the existing browser test pages, and the local Rollup migration branch described in the issue. Review the planned Karma and Travis work, then define completion as tests running in real browsers and ES5 environments, including the intended browser and IE coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs, rollup
Domain
ci-cd, frontend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.