openwall / openwall/john

Problem with --format=django and ~1m hashes

Open
#4,176 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

I'd like to audit django hashes. For a relatively small number of hashes - for example 50k - this work sfine. If I try to load ~1m records, things seem to go wrong. Loading takes forever / cracking never starts.

Steps to reproduce

john --format=django django_hashes.1m

Slow and dirty hash generation script (I'm sure there are faster ways ;-):

for i in {{1..1000000}}; do echo '$django$*1*pbkdf2_sha256$1$'`head -c8 /dev/urandom|base64`'$pO+MKfZvf0PfgRiMRtEZOH65l+6BJs/lKJvZ9P4aQsA='; done > django_hashes.1m
System configuration

Version: 1.9.0-jumbo-1+bleeding-6b14c5a 2019-11-15 14:46:38 +0100
Build: linux-gnu 64-bit x86_64 AVX2 AC OMP
SIMD: AVX2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: AVX2
$JOHN is ./
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 32768
SINGLE_BUF_MAX: 4294967295
Effective limit: Max. KPC 32768
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 5.4.0
GNU libc version: 2.23 (loaded: 2.23)
Crypto library: OpenSSL
OpenSSL library version: 01000207f
OpenSSL 1.0.2g 1 Mar 2016
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer claimed resolution 1 ns, observed 397 ns
Total physical host memory: 125 GB
Available physical host memory: 124 GB

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with John's --format=django loading path and reproduce the issue using the provided command and 1-million-record generator. Trace why loading does not reach cracking for the large input, then verify that the same workload loads and begins cracking without the reported delay.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, django
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.