openwall / openwall/john

Full UTF-32 support through-out Jumbo

Open
#1,632 25 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement non-trivial RFC / discussion
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Sub issues

  • #1627 (Incremental)
  • #1628 (Rules)
  • #1629 (Mask)
  • #1631 (FMT_UTF16)
  • #1641 (external)
  • #1642 (wordlist)
  • Subsets mode
  • none yet: Markov mode
  • none yet: (code page <-> unicode converter) Note: MB code pages are complex!

I'm trying to see the whole picture here. Let's say we start with saying that crk_set_key() takes UTF-32. This means all modes need to feed UTF-32 to it.

In that one place in cracker.c, the key will be converted to the target encoding (except for #1631).

After this, we'll just have to get all modes working. All of them should produce UTF-32. For Incremental, Mask and Markov, it's easy. Prince mode is somewhat special (it actually handles UTF-8 without a flaw, by design) so let's deal with it later. Wordlist mode should convert to UTF-32 upon reading the file. From that point (including rules engine) we're all UTF-32 right until format's set_key().

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cracker.c and the crk_set_key() entry point to understand the proposed UTF-32 flow. Then inspect the Incremental, Mask, Markov, Prince, Wordlist, Rules, and format set_key() paths, along with the listed sub-issues. Done requires all modes to produce UTF-32 and the target encoding conversion to work across the affected formats.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.