openwall / openwall/john

MacOS ntlmv2-opencl failure w/ Intel Iris Plus 640

Open
#3,259 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

notes/external issues
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

As best I can tell this issue is not covered by issue 3235, 3234 or even 3245. If I'm wrong please do let me know! Also, this bug does NOT exist in vanilla jumbo1 (installed via homebrew), only in the current git release.

Steps to reproduce

Use case 1 (working):

$brew install john-jumbo
$john --format=ntlmv2-opencl --devices=1 hashfile
Device 1: Intel(R) Iris(TM) Plus Graphics 640
Local worksize (LWS) 64, global worksize (GWS) 4194304
Loaded 19 password hashes with 19 different salts (ntlmv2-opencl, NTLMv2 C/R [MD4 HMAC-MD5 OpenCL])
Remaining 14 password hashes with 14 different salts
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:06  3/3 0g/s 51127p/s 2867Kc/s 2867KC/s 123456..colishot
...

Use case 2 (failure):
Clone this git repo, cd to src/ dir.

$brew remove john-jumbo
$brew install gcc openssl
$./configure CC="gcc-8" CPPFLAGS="-I/usr/local/opt/openssl/include" LDFLAGS="-L/usr/local/opt/openssl/lib" --prefix="/usr/local" --disable-native-tests
$make -s clean && make -sj4
$make install
$mkdir /usr/local/share/john
$cp -a ../run/* /usr/local/share/john/
$ln -s /usr/local/share/john/john /usr/local/bin/
$john --format=ntlmv2-opencl --devices=1 hashfile
Device 1: Intel(R) Iris(TM) Plus Graphics 640
Using default input encoding: UTF-8
Loaded 19 password hashes with 19 different salts (ntlmv2-opencl, NTLMv2 C/R [MD4 HMAC-MD5 OpenCL])
Remaining 11 password hashes with 11 different salts
Error creating binary file ~/.john/kernels/ntlmv2_kernel.cl_4ef2dbbfda0c0e69fedbfa0efac43970.bin: No such file or directory
Self test failed (cmp_one(0))
$john --test --format=ntlmv2-opencl --devices=1
Device 1: Intel(R) Iris(TM) Plus Graphics 640
Benchmarking: ntlmv2-opencl, NTLMv2 C/R [MD4 HMAC-MD5 OpenCL]... Error creating binary file ~/.john/kernels/ntlmv2_kernel.cl_541ab61321192a8bfe085a85e597ceeb.bin: No such file or directory
DONE
Note: This format may also be benchmarked using --mask (see doc/MASK).
Raw:	52690K c/s real, 752727K c/s virtual

(doesn't appear to matter if --disable-native-tests is supplied to configure or not, results are the same. I did it because other versions needed it to build properly)

Use case 3 (partial failure):
Same as use case 2, but I don't do systemwide (configure/compile without --prefix) and just run it directly from the ../run/ directory. In this case I don't get the error about creating the binary file. When I run john --test --format=ntlmv2-opencl --devices=1 it works fine. But when I try to run it on actual hashes, it still fails with "Self test failed", just without the error about creating the binary file.

System configuration
$ john --list=build-info
Version: 1.8.0.13-jumbo-1-bleeding-0d3ea928c 2018-05-21 23:35:49 +0200
Build: darwin16.7.0 64-bit x86_64 SSE2 AC OMP
SIMD: SSE2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
System-wide exec: /usr/local/bin
System-wide home: /usr/local/share/john
Private home: ~/.john
$JOHN is /usr/local/share/john/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 8.1.0
OpenCL headers version: 1.2
Crypto library: OpenSSL
OpenSSL library version: 0100020ff
OpenSSL 1.0.2o  27 Mar 2018
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
$ john --list=opencl-devices
Platform #0 name: Apple, version: OpenCL 1.2 (Nov 13 2017 23:06:50)
    Device #0 (0) name:     Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
    Device vendor:          Intel
    Device type:            CPU (LE)
    Device version:         OpenCL 1.2 
    Driver version:         1.1 
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 16, short 8, int 4, long 2
    Global Memory:          8 GB
    Global Memory Cache:    64 B
    Local Memory:           32 KB (Global)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 2 GB
    Max clock (MHz):        2300
    Profiling timer res.:   1 ns
    Max Work Group Size:    1024
    Parallel compute cores: 4
    Speed index:            36800

    Device #1 (1) name:     Intel(R) Iris(TM) Plus Graphics 640
    Device vendor:          Intel Inc.
    Device type:            GPU (LE)
    Device version:         OpenCL 1.2 
    Driver version:         1.2(Jan 30 2018 12:35:24) 
    Native vector widths:   char 1, short 1, int 1, long 1
    Preferred vector width: char 1, short 1, int 1, long 1
    Global Memory:          1.5 GB
    Local Memory:           64 KB (Local)
    Constant Buffer size:   64 KB
    Max memory alloc. size: 384 MB
    Max clock (MHz):        1000
    Profiling timer res.:   80 ns
    Max Work Group Size:    256
    Parallel compute cores: 48
    Stream processors:      384  (48 x 8)
    Speed index:            384000

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the ntlmv2-opencl commands in the issue on the Intel Iris Plus 640, comparing the Homebrew jumbo1 build with the current git build. Inspect the OpenCL execution and ~/.john/kernels binary-cache behavior described in the logs. Done means actual hashes pass the self-test and the kernel binary is created successfully in both installed and local builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, macos
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.