no-loader-dupecheck can have ridiculous performance impact

At first I thought we had a rare infinite loop condition on processing of successful guesses, but no - the loop actually does terminate eventually. Here it took 47 seconds to remove 67k instances of one hash, and during this time the program was unresponsive:

$ rm tmp.pot; ./john -no-log -se=tmp -pot=tmp.pot -form=nt -mask=123456 -verb=1 -no-loader-dupecheck pp8-sample-10m
No dupe-checking performed when loading hashes.
Using default input encoding: UTF-8
Loaded 10000000 password hashes with no different salts (NT [MD4 128/128 AVX 4x3])
Press 'q' or Ctrl-C to abort, almost any other key for status
66648g 0:00:00:47  1412g/s 0.02118p/s 0.02118c/s 211864C/s 123456
Session completed.

This is actually a reduced test case - I first saw the problem by trying to run actual cracking on a 100M file (not 10M above), and it appeared to get stuck right away for many minutes after loading, with no visible progress.

Somehow on a similar 1M file, the problem isn't seen - it's instant:

$ rm tmp.pot; ./john -no-log -se=tmp -pot=tmp.pot -form=nt -mask=123456 -verb=1 -no-loader-dupecheck pp8-sample-taken-1
No dupe-checking performed when loading hashes.
Using default input encoding: UTF-8
Loaded 1000000 password hashes with no different salts (NT [MD4 128/128 AVX 4x3])
Press 'q' or Ctrl-C to abort, almost any other key for status
6617g 0:00:00:00  12253g/s 1.851p/s 1.851c/s 1851KC/s 123456
Session completed. 

Similarly, actual cracking on the 1M file just works, with no apparent slowdown.

I am puzzled by this, even though I wrote the code for removal of successful guesses.

Start by reproducing the commands using -no-loader-dupecheck with pp8-sample-10m and pp8-sample-taken-1, then trace the successful-guess removal path described in the issue. Done means the 10M case no longer becomes unresponsive while the existing 1M behavior remains fast.