SQLite deleted-record / WAL recovery: evaluate a shared ALEAPP capability
- Lingua principale
- Python
- Stelle
- 908
- Fork
- 244
- Merge medio
- 10h 18m
- PR unite (30g)
- 288
Descrizione
### Background
#956 (Samsung Honeyboard clipboard) added a self-contained SQLite **WAL frame B-tree parser** (`_WalRecovery`) that recovers *deleted clipboard entries* directly from `ClipItem.db-wal` with no external dependencies. It's validated (decodes real records, e.g. `frame 8 / page 4`), but it currently lives inside that one artifact.
This issue is to decide whether ALEAPP should grow a **shared** SQLite-recovery capability, since many SQLite-backed artifacts (messages, browser history, call logs, …) have WALs / free pages holding deleted or un-checkpointed rows that a normal `SELECT` misses.
### Current situation
- ALEAPP has **no shared SQLite deleted-record / WAL recovery helper** today — artifacts that want it must roll their own (as #956 did).
- ALEAPP **already vendors CCL Forensics open code**: `abxread`/`checkabx` in `ilapfuncs.py`, and `scripts/ccl/` (`ccl_leveldb`, `ccl_protobuff`, `ccl_simplesnappy`, `ccl_android_fcm_queued_messages`). Vendoring CCL-authored open modules is an established pattern here — but see below re: SQLite specifically.
### The `_WalRecovery` parser from #956 — scope & caveats
Walks WAL frames → last-frame-wins per page → parses table-leaf B-tree pages (`0x0D`) → decodes cell payloads to column value lists. Caveats that matter for a general helper:
- **WAL-only.** Recovers records living in WAL frames (un-checkpointed / superseded state). It does **not** carve truly-deleted records from freelist pages / unallocated cell space / freeblocks in the main DB — the more common "deleted record recovery."
- **Leaf table pages only** (`0x0D`); no interior or index pages.
- **No overflow-page reassembly** — long rows truncate.
- Returns **positional** column lists — a shared helper would want table/schema awareness so callers get named columns.
### Does CCL tooling help here?
CCL / Alex Caithness are the reference authority on SQLite WAL & freespace forensics ([Time travelling with SQLite Journals and WAL](https://www.cclsolutionsgroup.com/post/time-travelling-with-sqlite-journals-and-wal), [The Forensic Implications of SQLite's WAL](https://digitalinvestigation.wordpress.com/2012/05/04/the-forensic-implications-of-sqlites-write-ahead-log/)). **However, their SQLite deleted-data recovery ships as commercial products — Epilog and [RabbitHole](https://www.cclsolutionsgroup.com/forensic-products/rabbithole) (v3 recovers deleted SQLite records) — not as an open-source module** we could vendor the way we vendor their ABX / LevelDB code. So CCL is a design reference here, not a drop-in library.
### Open-source options to evaluate
- **[sqlite-dissect](https://github.com/dod-cyber-crime-center/sqlite-dissect)** (DoD DC3) — the most complete open option: WAL, rollback journal, freelist, unallocated, freeblocks. Heavier; worth vetting for license/packaging fit.
- Smaller / targeted Python: [mdegrazia/SQLite-Deleted-Records-Parser](https://github.com/mdegrazia/SQLite-Deleted-Records-Parser) (freeblock carving), [dutchcoders/forensics-sqlite](https://github.com/dutchcoders/forensics-sqlite) (WAL frame dumping), [aramosf/recoversqlite](https://github.com/aramosf/recoversqlite); [fqlite](https://github.com/pawlaszczyk/fqlite) (Java).
### Assessment / options
1. **Keep it artifact-local** (status quo) — `_WalRecovery` stays in Honeyboard; others copy/adapt. Zero framework risk, but duplication.
2. **Generalize the in-house WAL reader** into an `ilapfuncs` helper (e.g. `recover_wal_records(db_path, table)` → named-column rows), ideally adding overflow handling. Lightweight, no new deps — but still **WAL-only** (won't recover checkpointed-deleted rows from freespace).
3. **Adopt an open library** (sqlite-dissect the leading candidate) for fuller recovery (WAL + journal + freespace). Broader coverage, but a real dependency + integration cost to vet.
**Suggested direction:** option **2** as a near-term shared helper — it's small, dependency-free, and covers the common WAL case #956 already proved — while tracking option **3** (sqlite-dissect) for when artifacts need freespace/freeblock carving beyond the WAL. CCL's commercial tools are out of scope for open-source integration but remain the design reference.
_Follow-up to #956; no code change proposed here yet._
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.