airvzxf / airvzxf/voxora

fix(hf): dest.is_file() skip-if-cached accepts truncated partial downloads

Open
#186 1 comment 0 reactions 0 assignees View on GitHub
bug pending-human
Dominant language
Rust
Stars
0
Forks
1
Avg merge
11m
Merged PRs (30d)
77

Description

## Problem

The resume path at `voxora-hf/src/source.rs:632-633` uses `if dest.is_file() { continue; }` to decide whether to re-download each file. A file that was killed mid-write and left on disk is indistinguishable from a completed download via this check. The `verify_sha256_sidecars` helper (also at `voxora-hf/src/source.rs:677-729`) only validates files that have a matching `.sha256` sidecar — most HF repos do not publish sidecars, so the check is a no-op for the typical case.

Reproduction: process A downloads shard 1 of a 4-shard model, crashes between `sync_all` (`voxora-hf/src/client.rs:170`) and `rename` (`voxora-hf/src/client.rs:183`); on-disk state is a truncated `shard-1-of-4.safetensors` with no `.partial` left. Process B starts `resolve`; `dest.is_file()` returns `true` for shard 1 → skip → the truncated shard is loaded. The next inference call returns garbage or panics.

## Recipe

Capture `Content-Length` from the HTTP response head, compare against `meta.len()` on the on-disk file, and re-download on mismatch. Also closes the cross-process "writer holds a half-written file" case where two processes race on the same cache directory (see related issue on advisory lock).

## Acceptance

- A test that writes a 1 KiB shard, truncates it to 256 B on disk, and asserts the next resolve re-downloads it.
- The skipped-file check uses `Content-Length` from the response head, not the on-disk file's existence.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.