openwall / openwall/john

Autoconf <-> Makefile overhaul

Open
#3,566 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug maintenance/cleanup
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

There's some terrifically b0rken behaviors we should fix (probably after next release) in our build process, in that several things happen in ./configure that definitely should be in make, and perhaps some case of vice versa (although I'm not sure I can think of anything).

Here's how it is supposed to be in my firm opinion:

  • ./configure should set things up for the system you build on. That is, check arch, CPU type, available libs, handle options like --with-openmpi or "systemwide" stuff and so on. Most is about the system and very little is about JtR Jumbo. It should definitely not handle anything about checking Jumbo plugins or dependencies (that's what a makefile is all about!).
  • make should merely build any files missing or not up to date according to whatever spec came from last ./configure.

If we get this straight and you do a git pull, ideally you shouldn't ever have to re-config or make clean. Only make. That's even if you got a few new plugin formats with that git update.
However, if you just installed a GPU and OpenCL (and didn't have that before) or decide to try out the rexgen lib, you'd need a re-conf and full rebuild.

To do:

  • The plugin handling done in ./configure should be in Makefile just like it was before we added autoconf to Jumbo. We got too excited and tried to move just about everything from Makefile to autoconf, and ended up with rather silly behavior ever since.
  • The dependency handling (currently plugin_deps.pl) happens in ./configure while it should happen during make. Moreover, I believe that script should possibly be ditched and we should learn how to do it with make and/or gcc themselves (I can't recall the details but a deps: target is often involved).
  • The creation of dynamic_big_crypt.c from dynamic_big_crypt_header.cin and dynamic_big_crypt_hash.cin happens in ./configure while it too should totally be in Makefile.
  • Probably some more things, add here...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the current plugin handling and dependency flow between ./configure and Makefile, including plugin_deps.pl. Then trace how dynamic_big_crypt.c is created from dynamic_big_crypt_header.cin and dynamic_big_crypt_hash.cin. Done means system setup remains in ./configure while plugin checks, dependencies, and generated files are handled during make without unnecessary reconfiguration or cleaning.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, shell
Domain
build-system
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.