microsoft / microsoft/TypeScript

Defer baseline comparisons until end of test

Open
#63,777 0 comments 1 reaction 1 assignee Assigned to @Copilot View on GitHub
Dominant language
Go
Stars
111k
Forks
14.3k
Avg merge
2d 4h
Merged PRs (30d)
132

Description

Currently if you re-enable all failing fourslash tests (by clearing out `failedtests.txt`) and run

```sh
npm run convertfourslash
hereby baseline-accept
```

You'll get a baseline file for `internal/fourslash/tests/gen/findReferencesAfterEdit_test.go`

However, if you run `npm run updatefailing`, that test is still disabled in `failedtests.txt`.

I think it is possibly due to a bug in the baselining logic. My guess is that baselining is currently too eager. Any test that requests multiple baselines where an edit has occurred in between probably fails because any of the following might be true:

1. the baselines were never accepted so the test could never succeed
2. the second baseline call will always read the results of the first baseline, and is assumed to fail
3. the second baseline was accepted, so the first call will always fail

I think we look at the original infrastructure for fourslash, every test actually compared against calculated baselines at the end of the test run. (I could be wrong though.)

_Originally posted by @DanielRosenwasser in https://github.com/microsoft/typescript-go/pull/1450#discussion_r2229480973_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.