Audit: latin-1 usage — 1 data-corruption bug on hashfile download + cleanups

Summary

A fan-out code review of every latin-1/encoding call site was done to answer "is latin-1 ever the right choice?" It is correct in exactly one place (lossless $HEX[...] byte handling) but is a real data-corruption bug on the hashfile download path and a couple of low-priority cleanups elsewhere.

Motivating case: international customer pwdumps whose cracked plaintext/usernames contain non-Latin-1 characters (emoji, CJK, Cyrillic).

1. HIGH — silent data corruption on hashfile download

hashview/hashfiles/routes.py:271

buf = io.BytesIO(body.encode('latin-1', errors='replace'))

body contains cracked plaintext/usernames stored as real Unicode (post hex→UTF-8 backfill). Any codepoint > U+00FF is silently replaced with ? — irreversible loss in the exported file, no error surfaced. errors='replace' masks the wrong-codec choice.

Fix: buf = io.BytesIO(body.encode('utf-8')) and set charset=utf-8 on the download send_file mimetype (~line 274).

2. LOW — misleading dead code

hashview/jobs/routes.py:726 — commented-out bytes.fromhex(c[1].username).decode('latin-1') contradicts the current UTF-8 plain-text storage contract; running it today would ValueError on non-hex usernames. The live path (jobs_assign_notification_hashes, ~routes.py:776-810) already handles this correctly.

Fix: Delete the commented block; optionally leave a one-line pointer to jobs_assign_notification_hashes.

3. INFO — consistency wart (not a bug)

hashview/auth/service.py:95, hashview/users/routes.py:190, hashview/users/routes.py:260 — .decode('latin-1') on bcrypt output. Bcrypt output is pure ASCII (verified: 0/300 random hashes had a byte > 0x7F), so this is byte-identical to utf-8 — zero runtime/security effect — but it misleadingly implies non-ASCII and diverges from hashview/setup/__init__.py:318 (utf-8).

Fix (optional, behavior-preserving): align the three sites to .decode('utf-8').

Sites confirmed correct — no change needed

• hashview/utils/utils.py:585 — latin-1 fallback decode of non-UTF-8 $HEX[...] bytes for length/char-class analysis. The one legitimate use: latin-1 is the only 8-bit codec that round-trips all 256 byte values 1:1, never crashes, never corrupts.
• hashview/utils/utils.py:457 & :488 — UTF-16LE + surrogatepass for NTLM/MSSQL hashing (correctly avoids latin-1).
• hashview/utils/utils.py:611 + hashview/api/routes.py:1834 — ingestion via surrogateescape + $HEX (correctly avoids latin-1; never crashes on international pwdumps).
• migrations/versions/a671bad25f89_fix_schema_drift.py:67 — mysql_default_charset='latin1' in downgrade() only, matching legacy schema for drift reconciliation.

Start with hashview/hashfiles/routes.py:271-274 and verify that downloaded international plaintext and usernames remain intact with a UTF-8 charset. Then review the commented block near hashview/jobs/routes.py:726 and the bcrypt decode sites in hashview/auth/service.py:95 and hashview/users/routes.py:190,260. Done means the corruption path uses UTF-8, misleading dead code is removed, and the optional decode cleanups are behavior-preserving.