libretro / libretro/libretro-database
Full-database audit — findings, verification method, and fix branches
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 877
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 24
Description
I ran a full audit of this repo: all of metadat/ (660 dats, ~727k game blocks) and cht/ (28,301 files, 44 systems). Everything below is condensed — the full reports, per-class verification verdicts and the scanning scripts are at https://github.com/ArjunKdaf/lrdb-audit (MIT — take anything useful). Posting this before I start sending fixes, so the volume doesn't come as a surprise and you can tell me where to start.
Method. Fresh clone of master (6fd53f9). Every dat parsed with a quote-aware clrmamepro parser — naive paren-counting silently corrupts on titles like Odekake Lester - Lelele no Le (^^; (Japan), worth knowing for any tooling on your side. Metadata checked against the union of no-intro/redump/tosec per platform. Cheat suspects were verified against the parsers of the cores that consume the files (Genesis Plus GX, PicoDrive, Gearsystem, Beetle PSX, PCSX-ReARMed, SwanStation, RetroArch's cheat_manager), cited to file and line at pinned commits, plus published code lists, Datel docs and this repo's git history — nothing is reported from pattern-matching alone. I used AI agents (Claude) for the legwork, same setup as my earlier PRs here; it's faster than doing it by hand and I have API budget to spare this month.
Wrong metadata — 10,191 orphaned CRCs, 4,423 stale titles, 1,140 duplicate CRCs database-wide:
- ~7,800 of the orphans are already fixed on my fork: machine-verified resyncs of N64, NES, Mega Drive, GBA, SNES, GB, SMS, GBC, Game Gear and NDS, same method as #1691. Held back deliberately — they'll come as single-platform PRs at whatever pace works for review.
- Cheapest remaining win: Sega 32X — 840 stale titles, 0 dead CRCs, a pure rename pass. Then Famicom Disk System, PC Engine, Jaguar, SG-1000 (700–1,200 findings each).
- ~4,200 blocks are orphaned by filename alone: N64 "(ByteSwapped)"/"(BigEndian)" esrb/franchise files, "Magnavox - Odyssey 2"/"Microsoft - MSX 2" space variants coexisting with canonical spellings.
- Small mechanical fixes: one corrupt dat (lost-level-archive Arduboy — a doubled quote, 6 of 29 games unparseable, one-character fix), 4 invalid hashes, 180 dats missing the final trailing newline, 1,202 byte-identical duplicate blocks, ~230 name fields with invisible NBSP/trailing-whitespace.
- One gap in my own open PR: #1691 doesn't cover NGPC releasemonth and serial (89 findings). Fold into the branch or follow-up — your call.
Missing metadata:
- PS1/PS2/PSP: ~33k metadata blocks are serial-keyed with no crc fields, so none of it joins into the RDBs — this is why PSP shows zero metadata despite 7 populated dats. A serial→crc backfill (or serial-aware join in the build tooling) lights up three platforms at once.
- 69 of 122 platforms have no metadata dats at all. Flagging the scale only — that's authoring work, not fixing.
Cheats — 1,462 confirmed defects, every class verified against core source:
- Actively harmful, worth fixing first: 2 Ristar codes under Game Gear that Gearsystem misroutes into unvalidated emulated-RAM writes, plus 3 misfiled Genesis codes whose correct copies already exist in the Genesis file.
- 848 PSX "Game Buster" codes fragmented into 4-char
+-pieces — dead in all three PSX cores. Mechanical re-join into canonical 8+4 form verified safe for all 848. - 924 Game Gear Game Genie codes with
+where cores require-— dead in GPGX/PicoDrive. The codes themselves are genuine; fix is mechanical. - 179 individually bad codes (letter-O-for-zero batches, illegal Game Genie letters, float-mangled values like
8095000000.00, prose in code fields), plus 89 N64 codes hit by one systematic value/address rotation bug — reconstruction verified line-by-line against gamehacking.org. - Systemic: all 752 fMSX files share one conversion bug (conflicting duplicate key); 3 files are wholly Mednafen-format so RetroArch loads zero cheats from them; 406 unparseable lines; 59 files declare wrong cheat counts.
- ~30,600 blank-code entries trace to the Feb 2020 bulk import (7cd0d683 and siblings — #994 already noted the gamehacking.org translator left entries blank). Restorable from the source. The separate 24k desc-only NDS entries are intentional folder headers, untouched.
- Deliberately not touched: 2 odd-looking but legitimate published XplorerPro codes, 8 codes unverifiable against any source, and 1 defect that exists verbatim at its published source (upstream of this repo).
Your call before anyone writes a branch: the retired KA ESRB rating on 1,322 entries (keep as historical or migrate?); 264 developer/publisher spelling clusters (Koei/KOEI at 254-vs-79 is the largest — normalize or leave?).
Core-side note: smsplus-gx has an empty retro_cheat_set — cheats never apply in that core regardless of data quality. Filing that upstream separately.
Plan. Single-topic fix branches, one PR at a time, at your review pace. If you'd rather I start somewhere specific, say so.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked audit reports and the metadat/ and cht/ sections, then agree with maintainers on one single-topic branch. Done means a narrowly scoped, independently verified fix is submitted as its own PR; the broader rating and spelling decisions remain unresolved.
Written by the indexing model from the issue text.
Assessment
- Domain
- data, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100