Offline downloads: PWA offline shell + local yt-dlp sync server

TinyTube Offline Downloads

Context

TinyTube plays videos exclusively via YouTube iframe embeds from a static GitHub Pages SPA — nothing works without internet. The goal: a kid's iPad (installed PWA) can play videos fully offline (plane/car trips), and a laptop running a local server works as a secondary scenario. This requires three things that don't exist today: a service worker (app shell offline), actual video files (yt-dlp download on the parent's laptop — personal use only, never committed/deployed), and on-device video storage + a native <video> playback path.

Selection is automatic, not per-video: a parent-gated setting "Offline playtime" (hours, default 0 = off). When > 0 and the sync server is reachable, the app keeps the first videos in gallery order (the existing gallerySort — what the kid would watch next) downloaded, taking videos until cumulative duration reaches the budget (the video that crosses the threshold is included, so a 2-hour video alone can satisfy a 2-hour budget). Videos that fall out of the target set (e.g. watched to completion) are evicted from device storage.

Wi-Fi-only downloads: Safari has no Network Information API, but the design makes this nearly automatic — sync only ever pulls from the laptop's LAN server, which is unreachable over cellular. As belt-and-braces, where navigator.connection exists (Chrome/Android) skip sync when connection.type === 'cellular'.

Hard constraint discovered during design: the deployed PWA is HTTPS, and Safari blocks fetch() to http://<lan-ip> (mixed content). So Wi-Fi sync requires the local server to be HTTPS with a cert the iPad trusts — mkcert, with a one-time root-CA install on the iPad. An AirDrop-import fallback (Phase 4) shares all the storage/playback code and needs no certs.

Architecture

laptop (same Wi-Fi)                          iPad PWA (https://pathikrit.github.io/TinyTube/)
make offline                                 Auto-sync (offline playtime budget > 0):
scraper/offline_server.py    ◀── HTTPS ───▶   POST /download, poll /videos, GET /file/<id>.mp4
 yt-dlp → downloads/<id>.mp4    (mkcert)      → cache.put into Cache Storage; evict out-of-set
                                             Service worker (vite-plugin-pwa): shell precache,
                                              videos.json NetworkFirst, thumbs/FA runtime cache
                                             Playback: cache.match → blob URL → <video>
                                              behind existing TouchShield/PausedOverlay

Storage choice: Cache Storage API (not OPFS/IndexedDB): cache.put(request, response) streams the download to disk without buffering the file in memory; playback via cache.match → response.blob() → URL.createObjectURL — WebKit cache blobs are disk-backed and Safari seeks blob URLs natively, so the SW never has to handle <video> range requests (a Safari minefield). navigator.storage.persist() + home-screen install exempts from Safari's 7-day eviction.

Data model:

• localStorage tinytube:offline:v1: { videos: { [id]: { title, channelTitle, channelId, duration, thumbnail, size, savedAt } } } — denormalized so offline gallery renders even after a video rotates out of videos.json. Bytes live in cache tinytube-videos-v1 keyed by synthetic same-origin URL ${BASE_URL}offline/<id>.mp4.
• Settings additions (useSettings.js DEFAULTS): syncServerUrl: '', offlineHours: 0 (0 = feature off).

Sync protocol (JSON over HTTPS, CORS allowlist https://pathikrit.github.io + http://localhost:5173):
GET /status (connectivity check) · POST /download {id} → 202 · GET /videos → {[id]: {state: queued|downloading|done|error, progress, size}} (panel polls ~2s) · GET /file/<id>.mp4 (Content-Type/Length, streamed into cache.put).

Phase 1 — PWA offline shell

• webapp/package.json: add vite-plugin-pwa (dev dep).
• webapp/vite.config.js: VitePWA({ registerType: 'autoUpdate', manifest: false /* keep public/manifest.webmanifest */, devOptions: { enabled: false } }). Workbox: precache built shell; globIgnores: ['videos.json'] (regenerated per deploy — must be runtime-cached, not precached); runtime routes: videos.json → NetworkFirst, i.ytimg.com → StaleWhileRevalidate (maxEntries ~600), ka-p.fontawesome.com → CacheFirst (icons offline); navigateFallback under base path.
• webapp/src/main.jsx: registerSW({ immediate: true }) from virtual:pwa-register.
• New webapp/src/hooks/useOnline.js: navigator.onLine + online/offline listeners.
• Deploy workflow needs no change (deploys dist/).
• Watch-progress bar (independent quick win, no offline dependency): in webapp/src/components/VideoCard.jsx (or wherever gallery thumbnails render), draw a thin red progress bar along the bottom edge of each thumbnail, width = pos/dur from watchStore.watched[video.id] (localStorage — YouTube provides no such data). Hidden when no entry; full-width for completed.

Deliverable: installed PWA opens gallery + thumbnails in airplane mode (playback still online-only); thumbnails show watch progress.

Phase 2 — Sync server + auto-sync

• New scraper/offline_server.py (same uv project; stdlib ThreadingHTTPServer + ssl, yt-dlp already a dep): download worker (1–2 concurrent), format bv*[vcodec^=avc1][height<=720]+ba[acodec^=mp4a]/b[ext=mp4] + merge_output_format: mp4 (Safari needs h264/aac), outtmpl downloads/%(id)s.mp4, progress hooks → in-memory state; rescan downloads/ on startup. Endpoints per protocol above.
• Makefile: make offline (runs server, auto-generates certs if missing, prints the URL to enter on the iPad); make offline-certs (checks mkcert, mkcert -install, cert with SANs for $(hostname).local + localhost + LAN IP into certs/; prints one-time iPad steps: AirDrop rootCA.pem → install profile → Certificate Trust Settings → full trust).
• .gitignore: downloads/, certs/.
• New webapp/src/lib/offlineStore.js: manifest load/persist, isDownloaded, saveVideo(id, meta, response) (cache.put + add thumbnail to thumbs cache), getVideoBlobUrl, removeVideo, usage(), requestPersist().
• New webapp/src/lib/syncClient.js: fetch wrapper for the 4 endpoints.
• New webapp/src/hooks/useOfflineStore.js: React state over offlineStore (mirrors useWatchStore style).
• New webapp/src/hooks/useOfflineSync.js — the auto-sync engine:
  • Computes the target set: walk the gallery-sorted video list (reuse gallerySort from useWatchStore.js), accumulate duration until ≥ offlineHours * 3600 (include the crossing video); skip videos with null duration.
  • Runs when: offlineHours > 0 && online && not navigator.connection?.type === 'cellular' && GET /status on syncServerUrl succeeds. Triggered on app load and when settings/watch-history change (debounced).
  • Diff against manifest: request missing ids via POST /download, poll /videos, stream completed files into offlineStore.saveVideo; evict downloaded ids not in the target set (offlineStore.removeVideo). Calls requestPersist() on first download.
• Settings.jsx — new "Offline" section (no separate panel/view): offline-hours input (0 = off, matching the existing draft/Save pattern), sync server URL field, live sync status line (server reachable · n of m synced · storage usage from estimate()).
• useSettings.js: syncServerUrl, offlineHours defaults + mutators.

Phase 3 — Local playback + offline gallery

• New webapp/src/components/LocalVideo.jsx: adapter with react-youtube's contract so VideoPlayer.jsx barely changes — onReady(e) where e.target wraps a <video playsInline> element with playVideo/pauseVideo/seekTo/getCurrentTime/getDuration; DOM events → YT codes (playing→1, waiting→3, pause→2, ended→0); blob URL from offlineStore, revoked on unmount.
• VideoPlayer.jsx (webapp/src/components/VideoPlayer.jsx:123): single branch — downloaded ? <LocalVideo> : <YouTube> (prefer local even when online). Save loop, resume seek, TouchShield, PausedOverlay, ControlsBar, ENDED→markCompleted→exit all stay shared. iOS autoplay block is already handled by PausedOverlay's Play button (user gesture).
• Offline gallery: when useOnline() is false, filter merged channels to downloaded ids (drop empty channels), append a synthesized channel for downloaded videos missing from videos.json (from manifest metadata); relax App's fatal-error screen when offline + manifest non-empty. If the downloaded set is empty, show a centered empty state: Offline <i class="fa-duotone fa-solid fa-signal-slash"></i>. VideoCard.jsx: downloaded badge (e.g. fa-circle-down) when online.

Phase 4 — Polish + AirDrop fallback

• Settings Offline section, import control: <input type="file" accept="video/mp4" multiple>, filenames <id>.mp4 matched to known videos → same cache.put path. Zero-cert escape hatch.
• "Clear offline videos" button + storage size shown; optional server DELETE.
• AGENTS.md: offline architecture, new storage keys/cache names, Make targets, mkcert setup, never-commit-videos rule. README.md: one line.

Verification

• Vitest (shared in-memory CacheStorage stub): offlineStore.test.js (save/list/remove); useOfflineSync target-set test (gallery order, hour-budget crossing, 0 = off, eviction diff); LocalVideo.test.jsx (DOM events → YT-code mapping, ref methods — mirrors existing VideoPlayer.test.jsx mock style); VideoPlayer.test.jsx additions (downloaded → LocalVideo, ENDED still marks watched + exits); gallery offline-filter + empty-state test; watch-progress-bar width test; syncClient.test.js (mocked fetch).
• Server smoke: curl -k https://localhost:8443/status; download one video end-to-end, verify h264/aac mp4 (ffprobe).
• Manual iPad script: certs + trust → install PWA → make offline → Settings: set offline hours to e.g. 2, enter server URL → watch auto-sync fill → airplane mode → relaunch: only downloaded videos show (thumbnails/icons render); with nothing downloaded, the "Offline" signal-slash empty state shows → play: resume/seek/pause, TouchShield + PausedOverlay intact, ENDED → watched + gallery → back online: badges + YouTube playback unaffected, completed video evicted + next one synced → reboot iPad, retest persistence.

Risks

Risk	Mitigation
Safari quota eviction	home-screen install (exempt from 7-day cap) + persist() + usage display
SW staleness / precached videos.json	autoUpdate + skipWaiting; videos.json excluded from precache, NetworkFirst
yt-dlp format gaps	fallback chain; ffmpeg on laptop; per-video error state in panel
mkcert friction / .local mDNS flakiness	cert SANs include LAN IP; make offline-certs regenerates; AirDrop fallback needs no certs

Start with Phase 1 in webapp/vite.config.js and webapp/src/main.jsx, then inspect the existing useWatchStore.js, VideoCard.jsx, and Vitest setup. Implementing the full issue spans the new offline hooks, storage and sync libraries, scraper/offline_server.py, local playback, settings, and service-worker behavior; done means the listed Vitest, server smoke, and iPad checks pass across all four phases.