dbNSFP: every documented download link is dead (S3 bucket no longer exists)
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 40
Description
Every download link on the dbNSFP page we document as the acquisition path is dead, so the documented route to obtaining dbNSFP cannot currently be followed by anyone.
Found while implementing the real drift probe in #320; kept out of that PR because it is a documentation/acquisition defect independent of drift.
## What is broken
The landing page itself is fine -- `https://sites.google.com/site/jpopgen/dbNSFP` returns 200 (~352 KB) and still advertises 64 distinct `dbNSFP*.zip` archives. **All of them 404**, because the bucket they point at does not exist:
```console
$ curl -sI https://dbnsfp.s3.amazonaws.com/dbNSFP4.9a.zip # newest academic release
HTTP/1.1 404 Not Found
$ curl -s "https://dbnsfp.s3.amazonaws.com/?list-type=2&max-keys=5"
NoSuchBucketThe specified bucket does not exist
dbnsfp...
```
Not an isolated stale link: `dbNSFP4.9c.zip`, `dbNSFP4.8a.zip`, `dbNSFP4.7a.zip`, `dbNSFP4.0a.zip` and `dbNSFP2.0.zip` were all checked and all return 404. `NoSuchBucket` means the whole S3 bucket is gone, not that individual objects were pruned.
The alternative mirror the same page names does not resolve at all:
```console
$ curl -sI -L --max-time 30 http://database.liulab.science/dbNSFP
# no response; exit 000
```
## Why it matters
dbNSFP is a documentation-only source by design (~45 GB, licence-gated, per the downloader strategy in CLAUDE.md), so the docs *are* the acquisition path. Right now they point somewhere that cannot work:
- [`docs_site/guide/data-sources.md:124-131`](docs_site/guide/data-sources.md#L124-L131) -- "**Download**: Requires academic license acceptance. Download from the project page", linking the Google Sites page.
- [`hvantk/skills/dbnsfp/SKILL.md:10`](hvantk/skills/dbnsfp/SKILL.md#L10) -- `Upstream: https://sites.google.com/site/jpopgen/dbNSFP`.
The pre-processing instructions further down `data-sources.md` reference concrete filenames (`dbNSFP4.9a_variant.chr*.gz`), so a user who cannot obtain the archive cannot reach any of it.
Note this does **not** affect the drift probe added in #320. That probe deliberately fingerprints the *release list the page advertises*, not the archives, precisely because the archives are unreachable; it is documented as detecting a release roll-over and explicitly not a file-level change. If the links are ever repaired, that is itself a change the probe will surface.
## What needs deciding
Someone needs to establish where dbNSFP is actually distributed now. Possibilities, none verified:
1. The maintainers moved distribution and have not updated the page (in which case: find the new host and update both docs).
2. Distribution is now fully gated behind a request/licence form with no public URLs (in which case: document the request process instead of a download link).
3. The resource is no longer actively distributed (in which case: mark the plugin's provenance accordingly and say so in `SKILL.md`).
I did not want to guess between these, since each implies a different fix and the wrong guess sends users somewhere unhelpful. Worth an upstream enquiry to the maintainers.
## Suggested acceptance criteria
- [ ] Determine the current, working acquisition route for dbNSFP 4.x.
- [ ] Update `docs_site/guide/data-sources.md` and `hvantk/skills/dbnsfp/SKILL.md` to match, or state plainly that no public route exists and what to do instead.
- [ ] Consider whether the docs guards (#313/#314) can assert that documented acquisition URLs still resolve, so the next breakage is caught rather than discovered.
The last point is the one with leverage: this failure was invisible until someone happened to fetch the links.
Refs #177, #320.
Contributor guide
Research direction
Start by reading docs_site/guide/data-sources.md:124-131, hvantk/skills/dbnsfp/SKILL.md:10, and the acquisition details in CLAUDE.md. Verify the listed URLs and contact or investigate the dbNSFP maintainers to establish the current route. Done means both documents describe a verified acquisition path, or plainly explain that no public route exists; assess whether guards #313/#314 should cover the documented URLs.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100