openwall / openwall/john

Move more DES-based formats to bitslice DES code

Open
#2,708 5 comments 0 reactions 1 assignee View on GitHub

@magnumripper is already working on this.

Since Dec 15, 2018.

enhancement
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

We currently use bitslice DES in these formats:

$ fgrep -l DES_bs.h *_fmt*.c
BSDI_fmt.c
DES_fmt.c
LM_fmt.c
MSCHAPv2_bs_fmt_plug.c
NETNTLM_bs_fmt_plug.c
opencl_DES_fmt_plug.c
trip_fmt.c

Two of them were moved to bitslice DES due to work by Deepika (thanks!)

$ fgrep -w Deepika *.c
DES_bs.c: * Deepika Dutta Mishra <dipikadutta at gmail.com> in
DES_bs_b.c: * Deepika Dutta Mishra <dipikadutta at gmail.com> in 2012, no
MSCHAPv2_bs_fmt_plug.c: * Modified for using Bitsliced DES by Deepika Dutta Mishra
NETNTLM_bs_fmt_plug.c: * Modified for using Bitsliced DES by Deepika Dutta Mishra

However, many of jumbo-specific formats continue to use OpenSSL's DES:

$ fgrep -l /des.h *_fmt*.c
KRB4_fmt_plug.c
KRB5_fmt_plug.c
NETLM_fmt_plug.c
NETSPLITLM_fmt_plug.c
as400_des_fmt_plug.c
dmg_fmt_plug.c
dpapimk_fmt_plug.c
itunes_fmt_plug.c
iwork_fmt_plug.c
keychain_fmt_plug.c
mozilla_ng_fmt_plug.c
ntlmv1_mschapv2_fmt_plug.c
o10glogon_fmt_plug.c
o3logon_fmt_plug.c
opencl_dmg_fmt_plug.c
opencl_gpg_fmt_plug.c
opencl_keychain_fmt_plug.c
opencl_zip_fmt_plug.c
oracle_fmt_plug.c
pem_fmt_plug.c
racf_fmt_plug.c
ssh_ng_fmt_plug.c
vnc_fmt_plug.c

Also, AFS_fmt.c inherited from core tree uses JtR's own non-bitslice DES code (only), but that's a relatively obscure format, so I don't care much. And KRB4_fmt_plug.c above appears to wrongly include DES_std.h, while not using anything from it (instead only using OpenSSL's DES) - if confirmed, we should drop that line to avoid further confusion. Other uses of DES_std.h in the list below are probably OK for now (they're special cases within formats that mainly use bitslice) - although we might get rid of some of them later as well:

$ fgrep -l DES_std.h *_fmt*.c
AFS_fmt.c
BSDI_fmt.c
DES_fmt.c
KRB4_fmt_plug.c
MSCHAPv2_bs_fmt_plug.c
NETNTLM_bs_fmt_plug.c
trip_fmt.c

We should identify non-obscure formats that still use solely/primarily non-bitslice DES where that matters for their overall performance, and proceed to move them to bitslice, on CPU for now. This should be easier than it used to be now that Deepika paved the way.

I think the Oracle (possibly including o*logon?), VNC, RACF, AS400 formats may be high on our list to switch to bitslice. (I was surprised to find Oracle wasn't switched to bitslice yet. IIRC, this was one of those Deepika worked on, but perhaps didn't finish the work? Or did we fail to merge a patch?)

A related issue is OpenSSL's dropping of DES support. This may force us to do something about uses of OpenSSL's DES code even where it doesn't matter for performance.

The above lists are for format files, but there are also uses in maybe-common files:

$ fgrep -l /des.h *.c | fgrep -v fmt
KRB4_std_plug.c
gpg_common_plug.c
mdc2dgst_plug.c
$ fgrep -l DES_std.h *.c | fgrep -v fmt
DES_bs.c
DES_std.c
ztex_descrypt.c

I think these are OK for now, but we'll need to revisit the first of these lists when dealing with OpenSSL's dropping of DES support.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.