diffplug / diffplug/selfie

[jvm] Snapshots are not being written to disk when there are tests with numbered names

Aperta
#540 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Kotlin
Stelle
101
Fork
18
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

There is an issue when creating test methods that are numbered (e.g. `test1()`, `test2()`, ...). At some point snapshots of tests are no longer written to disk. I discovered this while writing integration tests for a Spring Boot app that I generically name `test1()`, `test2()` etc. because I use `@DisplayName` anyway.

I cannot disclose the actual code but here is an example test file using JUnit 5, Java 21 and Selfie `2.5.3` that I could reproduce this with:

```java
import com.diffplug.selfie.Selfie;
import org.junit.jupiter.api.Test;

class ReproduceSelfieBugTest {
//SELFIEWRITE
@Test
void test() {
Selfie.expectSelfie("0").toMatchDisk();
}

@Test
void test1() {
Selfie.expectSelfie("1").toMatchDisk();
}

@Test
void test2() {
Selfie.expectSelfie("2").toMatchDisk();
}

@Test
void test3() {
Selfie.expectSelfie("3").toMatchDisk();
}

@Test
void test4() {
Selfie.expectSelfie("4").toMatchDisk();
}

@Test
void test5() {
Selfie.expectSelfie("5").toMatchDisk();
}

@Test
void test6() {
Selfie.expectSelfie("6").toMatchDisk();
}

@Test
void test7() {
Selfie.expectSelfie("7").toMatchDisk();
}

@Test
void test8() {
Selfie.expectSelfie("8").toMatchDisk();
}

@Test
void test9() {
Selfie.expectSelfie("9").toMatchDisk();
}

@Test
void test10() {
Selfie.expectSelfie("10").toMatchDisk();
}

@Test
void testFoo() {
Selfie.expectSelfie("testFoo").toMatchDisk();
}

@Test
void testFoo1() {
Selfie.expectSelfie("testFoo1").toMatchDisk();
}

@Test
void testFoo2() {
Selfie.expectSelfie("testFoo2").toMatchDisk();
}

@Test
void testFoo3() {
Selfie.expectSelfie("testFoo3").toMatchDisk();
}

@Test
void testFoo4() {
Selfie.expectSelfie("testFoo4").toMatchDisk();
}

@Test
void testFoo5() {
Selfie.expectSelfie("testFoo5").toMatchDisk();
}

@Test
void testFoo6() {
Selfie.expectSelfie("testFoo6").toMatchDisk();
}

@Test
void testFoo7() {
Selfie.expectSelfie("testFoo7").toMatchDisk();
}

@Test
void testFoo8() {
Selfie.expectSelfie("testFoo8").toMatchDisk();
}

@Test
void testFoo9() {
Selfie.expectSelfie("testFoo9").toMatchDisk();
}

@Test
void testFoo10() {
Selfie.expectSelfie("testFoo10").toMatchDisk();
}

@Test
void foo() {
Selfie.expectSelfie("foo").toMatchDisk();
}

@Test
void foo1() {
Selfie.expectSelfie("foo1").toMatchDisk();
}

@Test
void foo2() {
Selfie.expectSelfie("foo2").toMatchDisk();
}

@Test
void foo3() {
Selfie.expectSelfie("foo3").toMatchDisk();
}

@Test
void foo4() {
Selfie.expectSelfie("foo4").toMatchDisk();
}

@Test
void foo5() {
Selfie.expectSelfie("foo5").toMatchDisk();
}

@Test
void foo6() {
Selfie.expectSelfie("foo6").toMatchDisk();
}

@Test
void foo7() {
Selfie.expectSelfie("foo7").toMatchDisk();
}

@Test
void foo8() {
Selfie.expectSelfie("foo8").toMatchDisk();
}

@Test
void foo9() {
Selfie.expectSelfie("foo9").toMatchDisk();
}

@Test
void foo10() {
Selfie.expectSelfie("foo10").toMatchDisk();
}
}
```

Running this test file in IntelliJ 2024.3 produces this `ReproduceSelfieBugTest.ss` file:
```ss
╔═ foo ═╗
foo
╔═ foo1 ═╗
foo1
╔═ foo2 ═╗
foo2
╔═ foo3 ═╗
foo3
╔═ foo4 ═╗
foo4
╔═ foo5 ═╗
foo5
╔═ foo6 ═╗
foo6
╔═ foo7 ═╗
foo7
╔═ foo8 ═╗
foo8
╔═ foo9 ═╗
foo9
╔═ test ═╗
0
╔═ test1 ═╗
1
╔═ test2 ═╗
2
╔═ test3 ═╗
3
╔═ test4 ═╗
4
╔═ test5 ═╗
5
╔═ test6 ═╗
6
╔═ test7 ═╗
7
╔═ test8 ═╗
8
╔═ test9 ═╗
9
╔═ testFoo ═╗
testFoo
╔═ testFoo1 ═╗
testFoo1
╔═ testFoo2 ═╗
testFoo2
╔═ testFoo3 ═╗
testFoo3
╔═ testFoo4 ═╗
testFoo4
╔═ testFoo5 ═╗
testFoo5
╔═ testFoo6 ═╗
testFoo6
╔═ testFoo7 ═╗
testFoo7
╔═ testFoo8 ═╗
testFoo8
╔═ testFoo9 ═╗
testFoo9
╔═ [end of file] ═╗
```

You will notice the snapshots for these tests are missing: `foo10()`, `test10()`, `testFoo10`, i.e. all tests ending with `10`.

Additional information:
If you remove ALL tests except `test()`, `test1()` and `test10()` then the snapshot for `test10()` is _correctly_ saved to disk.
After that, if `test2()` is added back and after re-running the test file, the snapshot for `test10()` is **missing again**. Then, after removing `test2()` once more and re-running the test file, the snapshot for `test10()` is **STILL** missing until the test file is run **one more time**.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.