ChiselSim tests runs orders of magnitude slower than chiseltest

Converting some tests to ChiselSim, I've noticed the tests run about 20x slower than chiseltest.

As a comparison, running on ChiselSim:

```
❯ time ./mill chiselv.test.testOnly chiselv.ALUSpec
[86/86] chiselv.test.testOnly
ALUSpec:
- should ADD
- should ADDI
- should SUB
- should AND
- should ANDI
- should OR
- should ORI
- should XOR
- should XORI
- should SRA
- should SRAI
- should SRL
- should SRLI
- should SLL
- should SLLI
- should SLT
- should SLTI
- should SLTU
- should SLTIU
- should EQ
- should NEQ
- should GT
- should GTU
Run completed in 1 minute, 37 seconds.
Total number of tests run: 23
Suites: completed 1, aborted 0
Tests: succeeded 23, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
./mill chiselv.test.testOnly chiselv.ALUSpec 0.99s user 0.79s system 1% cpu 1:40.39 total
```

and on chiseltest:

```
❯ time ./mill chiselv.test.testOnly chiselv.ALUSpec
[86/86] chiselv.test.testOnly
ALUSpec:
- should ADD
- should ADDI
- should SUB
- should AND
- should ANDI
- should OR
- should ORI
- should XOR
- should XORI
- should SRA
- should SRAI
- should SRL
- should SRLI
- should SLL
- should SLLI
- should SLT
- should SLTI
- should SLTU
- should SLTIU
- should EQ
- should NEQ
- should GT
- should GTU
Run completed in 3 seconds, 661 milliseconds.
Total number of tests run: 23
Suites: completed 1, aborted 0
Tests: succeeded 23, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
./mill chiselv.test.testOnly chiselv.ALUSpec 0.31s user 0.10s system 8% cpu 4.846 total
```

I've followed the migration guide which ended with the following simple changes:

```diff
❯ gdpatch chiselv/test/src/ALUSpec.scala
diff --git a/chiselv/test/src/ALUSpec.scala b/chiselv/test/src/ALUSpec.scala
index ac56a81..b4bafd0 100644
--- a/chiselv/test/src/ALUSpec.scala
+++ b/chiselv/test/src/ALUSpec.scala
@@ -1,7 +1,7 @@
package chiselv

import chisel3._
-import chiseltest._
+import chisel3.simulator.EphemeralSimulator._
import com.carlosedp.riscvassembler.ObjectUtils.NumericManipulation
import org.scalatest._

@@ -9,7 +9,7 @@ import Instruction._
import flatspec._
import matchers._

-class ALUSpec extends AnyFlatSpec with ChiselScalatestTester with should.Matchers {
+class ALUSpec extends AnyFlatSpec with should.Matchers {
val one = BigInt(1)
val max = (one << 32) - one
val min_signed = one << 32 - 1
@@ -124,12 +124,12 @@ class ALUSpec extends AnyFlatSpec with ChiselScalatestTester with should.Matcher
dut.io.a.poke(i.to32Bit)
dut.io.b.poke(j.to32Bit)
dut.clock.step()
- dut.io.x.peekInt() should be(out)
+ dut.io.x.peek().litValue should be(out)
}
def testCycle(
op: Type
) =
- test(new ALU) { c =>
+ simulate(new ALU) { c =>
cases.foreach { i =>
cases.foreach { j =>
testDut(i, j, aluHelper(i, j, op).to32Bit, op, c)
```

The file is from https://github.com/carlosedp/chiselv/blob/main/chiselv/test/src/ALUSpec.scala

**Type of issue**: Bug Report

**Please tell us about your environment:**

Chisel 6.4.0 on MacOS Sonoma 14.5.
Verilator 5.024 2024-04-05 rev UNKNOWN.REV

Reproduce the comparison with ./mill chiselv.test.testOnly chiselv.ALUSpec using chiselv/test/src/ALUSpec.scala and the shown ChiselSim migration. Start by comparing the ChiselSim and chiseltest execution paths and their timing. Done means identifying and addressing the cause of the roughly 20x slowdown, with ALUSpec still passing and its runtime measured again.

Written by the indexing model from the issue text.