openwall / openwall/john

Duplicate rule suppression sometimes fails

Open
#5,011 26 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

As discussed in https://github.com/openwall/john/issues/5004#issuecomment-1019310089:

Shouldn't the current code have rejected a repeated rule like the below (when running -ru=jumbo)?

$ fgrep 'c Az"111"' sim3.log 
0:00:02:49 - Rule #923: '-c (?a c Az"111" <+' accepted as '(?acAz"111"<+'
0:00:02:49 - Score 0.012869018 at 10490.31 p/g 135g 1416192p during rule #923 :-c (?a c Az"111" <+
0:00:08:32 - Rule #2769: '-c (?a c Az"111" <+' accepted as '(?acAz"111"<+'
0:00:08:33 - Score 0.000000001 at 1416192000000000.00 p/g 0g 1416192p during rule #2769 :-c (?a c Az"111" <+

Obviously, it doesn't crack anything the second time this rule is tried. The log has this near the start:

0:00:00:00 - 72076 preprocessed word mangling rules were reduced by dropping 11 rules
0:00:00:00 - 72065 preprocessed word mangling rules

We should find out why the duplicate rule wasn't dropped.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the -ru=jumbo run and inspect the preprocessing output and sim3.log entries for rule #923 and rule #2769. Trace the duplicate-rule suppression path to determine why these normalized rules are both retained. Done means the repeated rule is dropped during preprocessing and the logs no longer show it being tried twice.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.