rspec-queue does not fail if there are NameErrors in specs

I've just encountered the following behaviour with the rspec runner of ci-queue:

1. Modify a spec to reference an uninitialized constant so as to trigger a NameError, e.g. by referring to a non-existent class:

describe MyNonExistentClass do
 ...
end

2. Run rspec-queue
3. rspec-queue exits successfully.

I guessed wrongly that rspec-queue would fail with a non-zero exit code, but it exited with 0.

Bare rspec exits with 1 in the above situation.

I'm wondering if this is intended behaviour - perhaps its needed for retries to work?

The error message output is:

NameError: uninitialized constant MyNonExistentClass

Debugging shows that line 400 is where the 0 exit code is returned, because the NameError causes @world.non_example_failure to be true. Syntax errors appear to have the same result:

https://github.com/Shopify/ci-queue/blob/cba2cddb6de4342fc7b5dd911f53d99651e3723e/ruby/lib/rspec/queue.rb#L387-L401

Inizia da ruby/lib/rspec/queue.rb intorno alle righe 387-401 e riproduci il caso NameError usando rspec-queue, confrontando il suo stato di uscita con quello di bare rspec. Traccia in che modo NameError e gli errori di sintassi influiscono sul risultato riportato e determina se il comportamento completato debba restituire uno stato diverso da zero senza interrompere i retry; il lavoro è completato quando sono presenti la copertura di regressione e lo stato di errore corretto.