openwall / openwall/john

Keystore format segmentation fault for 33+ threads

Open
#6,026 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

My setup is Ubuntu 26.4 + 2 x Intel(R) Xeon(R) 6787P
I see segmentation fault in keystore format self test when number of threads is greater than 32.
Up to 32 self test works as expected.

OMP_NUM_THREADS=33 ./john -test -format=keystore
Will run 33 OpenMP threads
Benchmarking: keystore, Java KeyStore [SHA1 512/512 AVX-512 16x]... (33xOMP) Segmentation fault         (core dumped) OMP_NUM_THREADS=33 ./john -test -format=keystore
OMP_NUM_THREADS=32 ./john -test -format=keystore
Will run 32 OpenMP threads
Benchmarking: keystore, Java KeyStore [SHA1 512/512 AVX-512 16x]... (32xOMP) DONE
Warning: "Many salts" test limited: 34/256
Many salts:     70249K c/s real, 2901K c/s virtual
Only one salt:  20763K c/s real, 2141K c/s virtual

GDB backtrace:

[Switching to Thread 0x7ffa5f3276c0 (LWP 22673)]
0x00007ffff7b879c1 in SHA1_Update () from /usr/lib/x86_64-linux-gnu/libcrypto.so.3
(gdb) bt
#0  0x00007ffff7b879c1 in SHA1_Update () from /usr/lib/x86_64-linux-gnu/libcrypto.so.3
#1  0x00005555557c1c55 in getPreKeyedHash (idx=<optimized out>) at keystore_fmt_plug.c:181
#2  crypt_all._omp_fn.0 () at keystore_fmt_plug.c:480
#3  0x00007ffff77c4b5f in ?? () from /usr/lib/x86_64-linux-gnu/libgomp.so.1
#4  0x00007ffff74a407a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:454
#5  0x00007ffff753772c in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Let me know if anything more is needed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with OMP_NUM_THREADS=32 and 33 using the keystore format self test. Inspect keystore_fmt_plug.c at getPreKeyedHash (line 181) and crypt_all (line 480), using the supplied GDB backtrace as the starting point. Done means the 33-thread self test completes without a segmentation fault while the 32-thread behavior remains working.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.