openwall / openwall/john

Consider dropping -Os from OPT_INLINE

Open
#5,883 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

core branch enhancement maintenance/cleanup notes/external issues regression RFC / discussion
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

We inherited this from core:

# Remove the "-Os" if you're using an ancient version of gcc
OPT_INLINE = -Os -funroll-loops -finline-functions

and we reimplement this in configure.ac:

   dnl now fill in extras for JtR's OPT_INLINE= value
   CFLAGS_EX=""
   JTR_FLAG_CHECK([-Os], 1)
   JTR_FLAG_CHECK([-funroll-loops], 0)
   JTR_FLAG_CHECK([-finline-functions], 1)
   AC_SUBST([OPT_INLINE_FLAGS],["${CFLAGS_EX}"])

Now I found that -Os actually hurts bitslice DES performance with gcc version 14.2.0 (Debian 14.2.0-19) when building for plain x86-64 SSE2 and running in a VM on Sandy Bridge-EP. Removing -Os doubles the performance there.

Looking at the generated code, with -Os gcc uses plain SSE (not SSE2) "floating-point" moves and bitwise instructions. Those were introduced earlier and are shorter, so this makes sense. I think I knew this would be happening before, and I think it usually didn't cause performance regressions before... although I also recall switching from SSE to SSE2 in the assembly source fairly early on (still in the Pentium 4 days) because some slowdown was otherwise seen somewhere (I don't recall the specifics, maybe it's in the commits history or an old core release announcement).

So -Os is risky. I don't know what regressions there would be now if we omit it. We should try, including benchmarking on a few systems.

On this system, this ~2x slowdown is only seen with gcc. clang is fine (uses SSE2 instructions despite of -Os, and the speed is fine as-is).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in configure.ac, where OPT_INLINE_FLAGS is assembled from the compiler flag checks. Build the bitslice DES code with and without -Os using gcc and clang, then benchmark it on several systems, including the reported x86-64 SSE2 case. Done means documenting the performance results and any regressions before deciding whether -Os can be removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
build-system, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.