ctengel / ctengel/simpler-objects
simple_download(): return server-reported digest instead of computing a local streaming hash
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Proposal
Make `simple_download()` **not** compute a local streaming SHA-256. Instead, return the digest the server reports in its response headers (or `None` if the server sent none), leaving any verification to the caller.
This is essentially the original `requests`-based behavior from objectindex's old `simple_download`, but with the bugs fixed.
### Current behavior
`simple_download()` streams the body through `hashlib.sha256()` (via a `WRITEFUNCTION` closure), then verifies the computed digest against the server's `Repr-Digest` header, raising `ClientError` on mismatch, and returns the **locally computed** digest (always 32 bytes, never `None`).
### Proposed behavior
- Drop the local hashing: stream straight to disk (`WRITEDATA=open_file` instead of a hashing `WRITEFUNCTION` closure).
- Parse the server digest from `Repr-Digest`, falling back to `Content-Digest`, and return it (`bytes` or `None`).
- No in-function verification (there is nothing locally computed to compare against).
- Keep `FOLLOWLOCATION`, the partial-file cleanup on pycurl/HTTP errors, and the `ClientError` raising for HTTP >= 400.
Return tuple stays `(digest, mime, sugg_fname, mtime)`; only `digest` changes meaning (server-reported, may be `None`).
### Bugs to fix vs. the original objectindex implementation
1. **Read the right header.** The old code read only `Content-Digest`, but this object server replies with `Repr-Digest` — so it effectively never received a digest from its own server. Read `Repr-Digest` (with `Content-Digest` as a fallback for arbitrary sites).
2. The objectindex caller already compares with `.hex()` and guards `if dl_cksum:` / `if checksum_val:`, so it tolerates a `None` digest.
### Caller impact (objectindex)
None required — both call sites already handle `None` and `.hex()`:
- `download()` still recomputes from disk via `file_checksum`, so it remains fully verified; the server-header compare becomes a cheap independent sanity check (no longer redundant with the disk recompute).
- `upload_remote()` passes the (possibly `None`) digest to `upload_core`, which recomputes from disk regardless and only asserts when a digest is present.
### Tradeoff
The downside is `upload_remote()` ingesting **arbitrary external URLs**: those rarely send a digest header, so there's no longer a wire-vs-disk integrity check during download (matches the original objectindex behavior). `download()` from this object server loses nothing because it always re-hashes the file. The win is dropping one redundant hash per flow (the streaming hash is the cheap one; the disk re-read remains either way).
### Tests
objectindex's `tests/test_simpler_objects.py` / `tests/test_client_io.py` are being made tolerant of **either** behavior (streaming-hash or header-only) so this can land without lockstep coordination.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.