abrignoni / abrignoni/ALEAPP

SQLite deleted-record / WAL recovery: evaluate a shared ALEAPP capability

未關閉
#957 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
908
分支
244
平均合併
10 小時 18 分鐘
30 天內合併 PR
288

描述

### Background

#956 (Samsung Honeyboard clipboard) added a self-contained SQLite **WAL frame B-tree parser** (`_WalRecovery`) that recovers *deleted clipboard entries* directly from `ClipItem.db-wal` with no external dependencies. It's validated (decodes real records, e.g. `frame 8 / page 4`), but it currently lives inside that one artifact.

This issue is to decide whether ALEAPP should grow a **shared** SQLite-recovery capability, since many SQLite-backed artifacts (messages, browser history, call logs, …) have WALs / free pages holding deleted or un-checkpointed rows that a normal `SELECT` misses.

### Current situation

- ALEAPP has **no shared SQLite deleted-record / WAL recovery helper** today — artifacts that want it must roll their own (as #956 did).
- ALEAPP **already vendors CCL Forensics open code**: `abxread`/`checkabx` in `ilapfuncs.py`, and `scripts/ccl/` (`ccl_leveldb`, `ccl_protobuff`, `ccl_simplesnappy`, `ccl_android_fcm_queued_messages`). Vendoring CCL-authored open modules is an established pattern here — but see below re: SQLite specifically.

### The `_WalRecovery` parser from #956 — scope & caveats

Walks WAL frames → last-frame-wins per page → parses table-leaf B-tree pages (`0x0D`) → decodes cell payloads to column value lists. Caveats that matter for a general helper:

- **WAL-only.** Recovers records living in WAL frames (un-checkpointed / superseded state). It does **not** carve truly-deleted records from freelist pages / unallocated cell space / freeblocks in the main DB — the more common "deleted record recovery."
- **Leaf table pages only** (`0x0D`); no interior or index pages.
- **No overflow-page reassembly** — long rows truncate.
- Returns **positional** column lists — a shared helper would want table/schema awareness so callers get named columns.

### Does CCL tooling help here?

CCL / Alex Caithness are the reference authority on SQLite WAL & freespace forensics ([Time travelling with SQLite Journals and WAL](https://www.cclsolutionsgroup.com/post/time-travelling-with-sqlite-journals-and-wal), [The Forensic Implications of SQLite's WAL](https://digitalinvestigation.wordpress.com/2012/05/04/the-forensic-implications-of-sqlites-write-ahead-log/)). **However, their SQLite deleted-data recovery ships as commercial products — Epilog and [RabbitHole](https://www.cclsolutionsgroup.com/forensic-products/rabbithole) (v3 recovers deleted SQLite records) — not as an open-source module** we could vendor the way we vendor their ABX / LevelDB code. So CCL is a design reference here, not a drop-in library.

### Open-source options to evaluate

- **[sqlite-dissect](https://github.com/dod-cyber-crime-center/sqlite-dissect)** (DoD DC3) — the most complete open option: WAL, rollback journal, freelist, unallocated, freeblocks. Heavier; worth vetting for license/packaging fit.
- Smaller / targeted Python: [mdegrazia/SQLite-Deleted-Records-Parser](https://github.com/mdegrazia/SQLite-Deleted-Records-Parser) (freeblock carving), [dutchcoders/forensics-sqlite](https://github.com/dutchcoders/forensics-sqlite) (WAL frame dumping), [aramosf/recoversqlite](https://github.com/aramosf/recoversqlite); [fqlite](https://github.com/pawlaszczyk/fqlite) (Java).

### Assessment / options

1. **Keep it artifact-local** (status quo) — `_WalRecovery` stays in Honeyboard; others copy/adapt. Zero framework risk, but duplication.
2. **Generalize the in-house WAL reader** into an `ilapfuncs` helper (e.g. `recover_wal_records(db_path, table)` → named-column rows), ideally adding overflow handling. Lightweight, no new deps — but still **WAL-only** (won't recover checkpointed-deleted rows from freespace).
3. **Adopt an open library** (sqlite-dissect the leading candidate) for fuller recovery (WAL + journal + freespace). Broader coverage, but a real dependency + integration cost to vet.

**Suggested direction:** option **2** as a near-term shared helper — it's small, dependency-free, and covers the common WAL case #956 already proved — while tracking option **3** (sqlite-dissect) for when artifacts need freespace/freeblock carving beyond the WAL. CCL's commercial tools are out of scope for open-source integration but remain the design reference.

_Follow-up to #956; no code change proposed here yet._

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。