Many tests fail with Mesa

Hello everyone. First of, here are some information

# ./john --list=build-info
Version: 1.9.0-jumbo-1+bleeding-ea33667eb 2019-09-23 12:43:34 +0200
Build: linux-gnu 64-bit x86_64 SSE2 AC OMP
SIMD: SSE2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
$JOHN is ./
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 2147483648
SINGLE_BUF_MAX: 4294967295
Effective limit: Number of salts vs. SingleMaxBufferSize
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 8.3.0
GNU libc version: 2.28 (loaded: 2.28)
OpenCL headers version: 2.0
Crypto library: OpenSSL
OpenSSL library version: 01010104f
OpenSSL 1.1.1d  10 Sep 2019
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer claimed resolution 1 ns, observed 1 us
Total physical host memory: 3947 MB
Available physical host memory: 3353 MB

# ./john --list=opencl-devices
Platform #0 name: Clover, version: OpenCL 1.1 Mesa 18.3.6
    Device #0 (1) name:     AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
    Device vendor:          AMD
    Device type:            GPU (LE)
    Device version:         OpenCL 1.1 Mesa 18.3.6
    Driver version:         18.3.6
    Native vector widths:   char 16, short 8, int 4, long 2
    Preferred vector width: char 16, short 8, int 4, long 2
    Global Memory:          2043 MB
    Local Memory:           32 KB (Local)
    Constant Buffer size:   716 MB
    Max memory alloc. size: 716 MB
    Max clock (MHz):        900
    Max Work Group Size:    256
    Parallel compute cores: 16
    Stream processors:      1024  (16 x 64)
    Speed index:            921600

After having installed what in my head looks like the requirements to provide opencl support for my Saphire HD 7850, john failes on opencl tests. More specifically, rar-opencl seems to break the test loop altogether.

# ./john --test --format=opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Benchmarking: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_sha1crypt_fmt_plug.c:277 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: KeePass-opencl [SHA256 AES/Twofish/ChaCha OpenCL]... Error: 74 unsupported relocations
Error: 74 unsupported relocations
Error: 74 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_keepass_fmt_plug.c:263 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: oldoffice-opencl, MS Office <= 2003 [MD5/SHA1 RC4 OpenCL/mask accel]... Error: 4 unsupported relocations
FAILED (cmp_one(0))
Benchmarking: PBKDF2-HMAC-MD4-opencl [PBKDF2-MD4 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_md4_fmt_plug.c:279 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: PBKDF2-HMAC-MD5-opencl [PBKDF2-MD5 OpenCL 4x]... Error: 4 unsupported relocations
Error: 4 unsupported relocations
Error: 4 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_md5_fmt_plug.c:276 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: PBKDF2-HMAC-SHA1-opencl [PBKDF2-SHA1 OpenCL 4x]... Error: 2 unsupported relocations
Error: 2 unsupported relocations
Error: 2 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_pbkdf2_hmac_sha1_fmt_plug.c:300 - Error running loop kernel
FAILED (cmp_all(-1))
Benchmarking: rar-opencl, RAR3 (length 5) [SHA1 OpenCL AES]... (2xOMP) Error: 6 unsupported relocations
Error: 6 unsupported relocations
Error: 6 unsupported relocations
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_rar_fmt_plug.c:334 - Error running loop kernel
FAILED (cmp_all(-1))
0: OpenCL CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST (-14) error in opencl_rar_fmt_plug.c:220 - Error releasing memory mappings

Here is another one for testing nt-opencl:

# ./john --test --format=nt-opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Benchmarking: NT-opencl [MD4 OpenCL/mask accel]... Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

0: OpenCL CL_INVALID_EVENT (-58) error in opencl_common.c:1801 - clWaitForEvents

And when I tried against a small test file of ntlm hashes:

# ./john ~/test-ntlm.txt --format=nt-opencl
Device 1: AMD PITCAIRN (DRM 2.50.0, 4.19.0-6-amd64, LLVM 7.0.1)
Using default input encoding: UTF-8
Loaded 32 password hashes with no different salts (NT-opencl [MD4 OpenCL])
Build log: input.cl:332:18: warning: unknown attribute 'max_constant_size' ignored

Self test failed (cmp_one(0))

Version of john is latest jumbo cloned from github. I cannot figure out if it is an issue with my installation, or from the related issues I've seen with previous versions. I remain at your disposal should you require any further information regarding the issue.

Start by reproducing ./john --test --format=opencl and the nt-opencl command on the reported Mesa environment. Read opencl_rar_fmt_plug.c, the other format files named in the errors, and opencl_common.c; done means identifying the Mesa-specific failure and establishing whether the affected OpenCL self-tests can run correctly.