openwall / openwall/john

off-by-one write in utf16_to_utf8_r or misuse in enc_lc/enc_uc

Open
#5,029 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
13.6k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Working on #5027 I found that enc_lc can write one more byte than specified by dst_bufsize argument. Name suggests that it is ok to pass sizeof of a buffer as this argument, but it causes overflow. It happens only with --encoding=utf-8 because write happens in utf16_to_utf8_r.

I am not sure if it is a bug in utf16_to_utf8_r because respective argument there is called dst_len. It might be misuse of utf16_to_utf8_r in enc_lc/enc_uc. Does dst_len include terminating null byte?

(I am not sure of meaning of arguments. I saw already that base64_convert can put null byte at to_len position. BTW is it ok?)

The problem is kind of real: I can trigger similar crash through enc_uc in a dynamic format.

dynamic_fmt.c:

static char *HandleCase(char *cp, int caseType)
{
	static UTF8 dest[256];
[...]
			enc_uc(dest, sizeof(dest), (unsigned char*)cp, strlen(cp));
[...]
			enc_lc(dest, sizeof(dest), (unsigned char*)cp, strlen(cp));
$ python -c 'print "b" * 32' > t.pw
$ python -c 'print "\xce\x90" * 42 + "aaaaa"' > t.txt
$ ./run/john --format='dynamic=md5(uc($p))' t.pw --wordlist=t.txt
[...]
==23927==ERROR: AddressSanitizer: global-buffer-overflow on address 0x557a9c5e4c00 at pc 0x557a9c083cb1 bp 0x7fffff6ffcd0 sp 0x7fffff6ffcc8
WRITE of size 1 at 0x557a9c5e4c00 thread T0
    #0 0x557a9c083cb0 in utf16_to_utf8_r /home/user/john/src/unicode.c:673
    #1 0x557a9c08438d in utf16_to_enc_r /home/user/john/src/unicode.c:793
    #2 0x557a9c08a387 in enc_uc /home/user/john/src/unicode.c:1834
    #3 0x557a9bc397b9 in HandleCase /home/user/john/src/dynamic_fmt.c:8389
    #4 0x557a9bc39991 in set_key /home/user/john/src/dynamic_fmt.c:1315
    #5 0x557a9bfeb09c in process_key /home/user/john/src/cracker.c:1166
    #6 0x557a9c0551c7 in do_wordlist_crack /home/user/john/src/wordlist.c:1349
    #7 0x557a9c00822b in john_run /home/user/john/src/john.c:1809
    #8 0x557a9c00822b in main /home/user/john/src/john.c:2082
    #9 0x7fd2abd1109a in __libc_start_main ../csu/libc-start.c:308
    #10 0x557a9bb7e1f9 in _start (/home/user/john/run/john+0x1ed1f9)

0x557a9c5e4c00 is located 32 bytes to the left of global variable 'fncs' defined in 'dynamic_fmt.c:7199:33' (0x557a9c5e4c20) of size 160
0x557a9c5e4c00 is located 0 bytes to the right of global variable 'dest' defined in 'dynamic_fmt.c:8383:14' (0x557a9c5e4b00) of size 256
[...]

\xce\x90 (\u0390) becomes \xce\x99\xcc\x88\xcc\x81 during uppercasing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the AddressSanitizer failure through HandleCase in dynamic_fmt.c, then trace enc_uc and enc_lc through utf16_to_enc_r and utf16_to_utf8_r in unicode.c. Check how dst_bufsize and dst_len treat the terminating null byte, and verify the fix with the supplied dynamic format and UTF-8 input without a buffer overflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.