openwall / openwall/john

Add option to hybrid mask mode for expecting placeholders from parent

Open
#4,011 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Idea: Let hybrid parent mode feed candidates containing mask placeholders, to hybrid mask.

  1. For example, we build a PRINCE wordlist from the morphemes "secret", "1984" and "2019" but we change the two latter to just one instance of "?d?d?d?d". That doesn't mean anything special to PRINCE so we end up getting eg. "secret?d?d?d?d" and "?d?d?d?dsecret").
  2. Now, let's add an option to hybrid mask - let's say --mask=?p instead of ?w - that expects these placeholders within the parent mode's words, and expands them just like it would for eg. a base word of "secret" and a mask option of --mask=?w?d?d?d?d and then one of --mask=?d?d?d?d?w.

This can eg. be used for extracting unique base words ("morphemes") from a wordlist (or already cracked passwords) while replacing eg. all 4-digit numbers with a single entry of ?d?d?d?d, any specials with a single entry of ?s, and so on.

Pros:

  • PRINCE wordlist is kept much smaller and no rules (such as some massive "replacedigits" rule) needed.
  • We'll end up using fast hybrid mask (even ultra-fast device-side when applicable) instead of slow post-PRINCE rules!
  • Oh, and PRINCE is merely an example parent mode; This is all about mask mode. We can use this with wordlist or other modes as well.

Cons/caveats:

  • Current mask_init() (where placeholders and ranges are currently parsed) was never meant to be anywhere near the inner loops so might be fairly slow (not very hard to fix, I guess). If needed for performance we could opt to only support placeholders like ?d as opposed to ranges like [0-9] but I doubt it'll matter much.
  • Internal mask (re-)setup needs format's reset() to be called (in case of FMT_MASK), possibly triggering an auto-tune for (worst case) every parent word (see #4012 though). We can opt to run with fixed LWS/GWS or auto-tune for some compromise.
  • Furthermore, each and every time reset()needs to be called (per above), we first have to call crk_process_buffer() or any half-filled buffer will be lost/invalid. Consequently we may also get low buffer-fill and corresponding warnings.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading mask_init(), the hybrid mask and parent-mode paths, and the reset() and crk_process_buffer() interactions mentioned in the issue. Trace how FMT_MASK is handled and consider the implications noted in #4012. Done means a mask option can consume placeholders from parent words while preserving correct buffer processing and reset behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cli, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.