galaxyproject / galaxyproject/loom
Orbit: validate downloaded files' content type/signature; warn when extension doesn't match actual content
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
## What happens
While hunting for a TF-target database, the agent downloaded several files whose **content didn't match their extension**, and didn't notice until much later:
- A `.xlsx` saved from a public article's supplementary-data URL was actually an HTML page; it was only caught when the file wouldn't open in Excel.
- A `.tsv.gz` fetched from one source was an HTML document (`file` reported "HTML document text"), so `gunzip` failed.
- Another `.tsv.gz` was an S3 `AccessDenied` XML body saved as if it were gzipped data.
In each case the download "succeeded" (HTTP 200, non-zero file written), so the agent moved on and only discovered the mismatch a few steps later when something downstream broke.
## Expected vs actual
- **Expected:** right after a download, validate the file against what it's supposed to be -- MIME type / magic bytes / a quick `file` check -- and warn when a file labeled `.xlsx`/`.gz`/`.tsv` is actually HTML or XML.
- **Actual:** the file is saved under the requested name with no content check; mislabeled downloads slip through silently.
## Suggested direction
A lightweight post-download sanity check would catch the whole class: sniff the first bytes (e.g. ` not the binary/table the agent asked for; PK magic for real xlsx; gzip magic `1f 8b` for `.gz`) and surface a warning so the agent re-fetches or picks another source instead of building on a bad file. Relates to #282 (prefer server-side fetch-from-URL).
Contributor guide
No contributing guide indexed for this repository
Research direction
No source files or tests are named. Start by locating Orbit's download entry point and tracing where downloaded files are saved; define the post-download warning behavior for mismatched HTML, XML, XLSX, TSV, and gzip content. Done means representative mismatches are detected and tested without blocking valid downloads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100