`@batch` slows down other non-@batched code on x86

Consider the following functions.
```julia
using Polyester

function reset_x!(x)
x .= 0
end

function without_batch(x)
for i in 1:100_000
if x[i] != 0
# This is never executed, x contains only zeros
sin(x[i])
end
end
end

function with_batch(x)
@batch for i in 1:100_000
if x[i] != 0
# This is never executed, x contains only zeros
sin(x[i])
end
end
end
```
Running `with_batch` between `reset_x!` calls seems to somehow slow down `reset_x!` significantly:
```
julia> x = zeros(Int, 1_000_000);

julia> using BenchmarkTools

julia> @benchmark reset_x!($x) setup=without_batch(x)
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 92.173 μs … 426.870 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 94.362 μs ┊ GC (median): 0.00%
Time (mean ± σ): 94.776 μs ± 3.789 μs ┊ GC (mean ± σ): 0.00% ± 0.00%

▂▄▅▆▇██████▇▇▆▅▅▄▃▂▂▂ ▁▁▁ ▁ ▁▁▁▁▁▁▁ ▃
▄▄▄▆███████████████████████████▇██▇▇█▇████████████▇▆▆▇▇▆▆▆▆▆ █
92.2 μs Histogram: log(frequency) by time 101 μs <

Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark reset_x!($x) setup=with_batch(x)
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 185.404 μs … 518.292 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 188.394 μs ┊ GC (median): 0.00%
Time (mean ± σ): 189.119 μs ± 4.086 μs ┊ GC (mean ± σ): 0.00% ± 0.00%

▅▇█▇▇▇▅▅▃▂
▁▁▁▂▃▆▇█████████████▇▆▅▅▄▄▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁ ▃
185 μs Histogram: frequency by time 198 μs <

Memory estimate: 0 bytes, allocs estimate: 0.

julia> versioninfo()
Julia Version 1.9.0
Commit 8e630552924 (2023-05-07 11:25 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 128 × AMD Ryzen Threadripper 3990X 64-Core Processor
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, znver2)
Threads: 8 on 128 virtual cores
```
I also tested and reproduced this on an Intel i9-10980XE, but the difference there was only ~10% on 8 threads.

CC: @sloede @ranocha @LasNikas.