openwall / openwall/john

OpenCL formats running post-process on CPU

Open
#3,242 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

See also #3216

These formats (and possibly a few more) should be reviewed for moving post-processing to GPU (and drop OMP) even though their GPU utilization is not too bad even running single-thread host:

  • Encfs [AES-CFB]
  • PGPdisk [AES/TwoFish/CAST ECB]
  • PGPSDA [CAST-ECB]
  • Ethereum [Keccak]. This one doesn't even use (or need) OpenMP for the little post-processing because the GPU part is so very slow. Should be fixed anyway for good measure.
  • RAR5 has some ridiculous CPU post-processing, super trivial to get GPU-side.
  • DiskCryptor [AES/TwoFish/Serpent XTS], should be trivial.

For reference, these few formats will likely be impossible to get fully GPU-side (I will not even try):

  • RAR (actually RAR3 non-hp) [AES-CBC, unrar]. There's no way anyone can implement unrar on GPU. However, GPU util is fine even running single-thread host due to excellent early rejection.
  • 7z [LzmaDecode]. Fairly good early-rejection and GPU utilization even on single-thread host.
  • Electrum-modern [BIGNUM, secp256k1 and zlib inflate]. GPU util is ~0% even with 32 threads OMP.
  • GPG [IDEA, CAST5, Camellia and BIGNUM]. GPU util is poor even with 32 threads OMP.
  • Dashlane [AES-CBC, zlib inflate]. GPU util is 73% running one CPU thread and Titan X.
  • Tezos [ed25519_publickey, blake2b]. GPU util is 32% with 8xOMP.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the related issue #3216 and locating the OpenCL implementations and CPU post-processing for DiskCryptor, RAR5, Ethereum, and the other listed formats. Review how their GPU and OpenMP paths are structured. Done means the applicable post-processing runs on the GPU and unnecessary OpenMP use is removed, with GPU utilization and correctness checked.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance, security
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.