galaxyproject / galaxyproject/loom

Orbit: validate downloaded files' content type/signature; warn when extension doesn't match actual content

Open
#356 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

## What happens

While hunting for a TF-target database, the agent downloaded several files whose **content didn't match their extension**, and didn't notice until much later:

- A `.xlsx` saved from a public article's supplementary-data URL was actually an HTML page; it was only caught when the file wouldn't open in Excel.
- A `.tsv.gz` fetched from one source was an HTML document (`file` reported "HTML document text"), so `gunzip` failed.
- Another `.tsv.gz` was an S3 `AccessDenied` XML body saved as if it were gzipped data.

In each case the download "succeeded" (HTTP 200, non-zero file written), so the agent moved on and only discovered the mismatch a few steps later when something downstream broke.

## Expected vs actual

- **Expected:** right after a download, validate the file against what it's supposed to be -- MIME type / magic bytes / a quick `file` check -- and warn when a file labeled `.xlsx`/`.gz`/`.tsv` is actually HTML or XML.
- **Actual:** the file is saved under the requested name with no content check; mislabeled downloads slip through silently.

## Suggested direction

A lightweight post-download sanity check would catch the whole class: sniff the first bytes (e.g. ` not the binary/table the agent asked for; PK magic for real xlsx; gzip magic `1f 8b` for `.gz`) and surface a warning so the agent re-fetches or picks another source instead of building on a bad file. Relates to #282 (prefer server-side fetch-from-URL).

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files or tests are named. Start by locating Orbit's download entry point and tracing where downloaded files are saved; define the post-download warning behavior for mismatched HTML, XML, XLSX, TSV, and gzip content. Done means representative mismatches are detected and tested without blocking valid downloads.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.