openwall / openwall/john

raw-SHA512-free-opencl and XSHA512-free-opencl sometimes fail self-test

Open
#4,434 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

On a build made with devtoolset-3 and --enable-asan:

[solar@super run]$ ./john -te -form=raw-SHA512-free-opencl -dev=1
NOTE: This is a debug build, speed will be lower than normal
Device 1:        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Benchmarking: raw-SHA512-free-opencl [SHA512 OpenCL (inefficient, development use mostly)]... Binary Build log: Linking started
Linking done
Kernel <kernel_sha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
LWS=256 GWS=4194304 (16384 blocks) DONE
Raw:    7913K c/s real, 653572 c/s virtual

[solar@super run]$ ./john -te -form=XSHA512-free-opencl -dev=1
NOTE: This is a debug build, speed will be lower than normal
Device 1:        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Benchmarking: XSHA512-free-opencl, Mac OS X 10.7+ [SHA512 OpenCL (efficient at "many salts" only)]... Binary Build log: Linking started
Linking done
Kernel <kernel_xsha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
FAILED (cmp_all(23))
[solar@super run]$ ./john -te -form=XSHA512-free-opencl -dev=1
NOTE: This is a debug build, speed will be lower than normal
Device 1:        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Benchmarking: XSHA512-free-opencl, Mac OS X 10.7+ [SHA512 OpenCL (efficient at "many salts" only)]... Binary Build log: Linking started
Linking done
Kernel <kernel_xsha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
LWS=128 GWS=2097152 (16384 blocks) DONE
Warning: "Many salts" test limited: 21/256
Many salts:     21275K c/s real, 679947 c/s virtual
Only one salt:  8346K c/s real, 640351 c/s virtual

[solar@super run]$ ./john -te -form=raw-SHA512-free-opencl -dev=1
NOTE: This is a debug build, speed will be lower than normal
Device 1:        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Benchmarking: raw-SHA512-free-opencl [SHA512 OpenCL (inefficient, development use mostly)]... Binary Build log: Linking started
Linking done
Kernel <kernel_sha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
FAILED (cmp_all(43))
[solar@super run]$ ./john -te -form=raw-SHA512-free-opencl -dev=1
NOTE: This is a debug build, speed will be lower than normal
Device 1:        Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Benchmarking: raw-SHA512-free-opencl [SHA512 OpenCL (inefficient, development use mostly)]... Binary Build log: Linking started
Linking done
Kernel <kernel_sha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
LWS=256 GWS=2097152 (8192 blocks) DONE
Raw:    7951K c/s real, 652302 c/s virtual

Also:

./john -test -v=5 -form=opencl -dev=1 2>&1 | tee -a log1

resulted in:

Benchmarking: raw-SHA512-free-opencl [SHA512 OpenCL (inefficient, development use mostly)]... Loaded 9 hashes with 1 different salts to test db from test vectors
Options used: -I opencl -cl-mad-enable -D__CPU__ -DDEVICE_INFO=33 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=2 -D_OPENCL_COMPILER -DPLAINTEXT_LENGTH=20 $JOHN/opencl/sha512_kernel.cl
Build log: Compilation started
Compilation done
Linking started
Linking done
Kernel <kernel_sha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
binary size 5114
LWS=7 GWS=49 (7 blocks) FAILED (cmp_all(14) $SHA512$fa585d89c851dd338a70dcf535aa2a92fee7836dd6aff1226583e88e0996293f16bc009c652826e0fc5c706695a03cddce372f139eff4d13959da6f1f5d3eabe)

and

Benchmarking: XSHA512-free-opencl, Mac OS X 10.7+ [SHA512 OpenCL (efficient at "many salts" only)]... Loaded 18 hashes with 16 different salts to test db from test vectors
Options used: -I opencl -cl-mad-enable -D__CPU__ -DDEVICE_INFO=33 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=1 -DDEV_VER_MINOR=2 -D_OPENCL_COMPILER -DPLAINTEXT_LENGTH=20 -DSALT_SIZE=4 $JOHN/opencl/xsha512_kernel.cl
Build log: Compilation started
Compilation done
Linking started
Linking done
Kernel <kernel_xsha512> was not vectorized
Kernel <kernel_cmp> was successfully vectorized
Done.
binary size 5214
LWS=7 GWS=49 (7 blocks) FAILED (cmp_all(8) $LION$52396836b22e1966f14f090fc611ed99916992d6e03bffa86fe77a4993bd0952e706c13acc34edefa97a1dee885c149b34c27b8b4f5b3b611d9e739833b21c5cf772e9e7)

BTW, with formats failing during auto-tune like this, there's no empty line before the next format's "Benchmarking ..." line, which makes it hard to notice the failures. I only noticed from the final:

2 out of 89 tests have FAILED

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failures with the reported -test commands and device settings, then inspect opencl/sha512_kernel.cl and opencl/xsha512_kernel.cl. Compare the intermittent self-test results and auto-tune output with passing formats; done means both formats pass reliably and failures remain clearly reported.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cryptography, performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.