ReactiveX / ReactiveX/rxjs

Proposal to improve marble testing

Open
#5,674 0 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
31.7k
Forks
3k
PR merge metrics
No merged PRs in 30d

Description

Feature Request

Marble tests could be improved to better communicate to users why a test fails

Is your feature request related to a problem? Please describe.

When a test fails, it doesn't tell you which line the error is on. Once you figure out (often via trial & error) which expectObservable call is failing, often the diff output takes hours to debug for more complex tests because it surfaces a big JSON diff in jest instead of using marbles to more clearly show the diff.

Describe the solution you'd like

  • When expectObservable is called, a stack trace will be preemptively captures, and stored in the testScheduler, when flush is called, any failing expectObservable call will have a stack trace from whene it was called, which can be surfaced in the event the expectation fails. A JS error could be thrown & caught, and the error itself queued with the expectation, later if the expectation fails, the queued error would be re-thrown.
  • When a failure occurs, an ASCII / marble diagram [optionally, could be opt in via env var or feature toggle] would be printed to the console, so long as the number of frames is smaller. If the number of frames is larger, time progression syntax would be used. Alternatively, time progression syntax could be printed to the console if it was detected that the user used time progression syntax in their assertion(s).

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

(If this is new operator request) describe reason it should be core operator
A clear and concise description of reason it can't be achieved via user-land module.

Additional context

Not clear where failure is occurring! Also not clear it says - Expected - 10 + Received + 1 where is the 1 coming from?? (the weird integers are unrelated to RxJS, but its still not clear where the failure happens which was my main point, here is a separate bug to track the weird jest output https://github.com/nrwl/nx/issues/3603)

Screen Shot 2020-08-24 at 9 09 46 AM

I added a console.trace() to expectObservable to show the information could be surfaced about where expectObservable was called. It could also throw & immediately catch an error to get a stack trace.

Screen Shot 2020-08-24 at 9 13 34 AM

For example of better ASCII based diff reporting, see what https://github.com/kwonoj/rx-sandbox does

PS > I can volunteer to make PRs for both of these proposed features.
PS > Also just noticed '(a' is considered valid, ideally it should check for balanced parens, if its not expected to match a literal value
PS > It would also be nice to check if there are extra items in the values object which aren't used in the marbles string, and throw a validation error if so

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing expectObservable and TestScheduler.flush to understand how failed expectations are currently reported. Review the proposed stack-trace capture and marble or time-progression diff output, using the existing failure behavior as the baseline. Done should make failures identify the expectObservable call and present a clearer diagram-based comparison; decide separately whether validation of parentheses and unused values belongs in scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.