HTML papers emit scheme-less hrefs (bare DOIs / emails / URLs) and relative figure paths that 404 under /html/<id>/…
- Dominant language
- No language data
- Stars
- 42
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
A large fraction of arXiv HTML papers emit **hyperlinks and image sources without a URL scheme (or as bare relative paths)**. Browsers and crawlers resolve these relative to the paper's own URL, so a link meant for `https://doi.org/10.1038/30918` is instead requested as `https://arxiv.org/html//10.1038/30918` and returns **404**.
This has two effects:
1. **Readers get broken links / missing figures** in the HTML rendering.
2. **Crawlers following these links generate very large volumes of `/html//…` 404s.** This surfaced while investigating a 404 traffic spike; a single 2-hour origin-log sample already contained **975 distinct broken `/html/` links across 310 distinct papers** — and that is only the slice reaching origin, so the true volume is much larger.
## Bug class A — external references emitted without a scheme
The href is written relative (no `https://` / `mailto:`), so it resolves under `/html//…` and 404s. All three below are verified against the live HTML:
| Witness article | Malformed `href` in the HTML | Resolves to (404) |
|---|---|---|
| [2403.15855v1](https://arxiv.org/html/2403.15855v1) | `href="10.1038/30918"` | `/html/2403.15855v1/10.1038/30918` |
| [2402.15814v2](https://arxiv.org/html/2402.15814v2) | `href="ryan.cotterell@inf.ethz.ch"` | `/html/2402.15814v2/ryan.cotterell@inf.ethz.ch` |
| [2311.06334v2](https://arxiv.org/html/2311.06334v2) | `href="www.tng-project.org/data"` | `/html/2311.06334v2/www.tng-project.org/data` |
Note: 2403.15855v1 contains **both** `href="10.1038/30918"` (broken) *and* `href="https://doi.org/10.1038/30918"` (correct) for the same DOI — so scheme-less DOI hrefs are emitted inconsistently.
More examples from the sample (same pattern):
- **Bare DOIs:** `2412.01920v2 → 10.1088/1367-2630/14/12/123011`, `2509.22198v1 → 10.1002/icd.657`, `2410.10125v1 → dx.doi.org/10.1016/j.cognition.2010.10.004`
- **Emails missing `mailto:`:** `2403.00139v1 → timleung@uw.edu`, `2206.05955v2 → zvi.shem-tov@mail.huji.ac.il`
- **Scheme-less URLs:** `2409.14411v2 → scaling-diffusion-policy.github.io`, `2405.14093v8 → www.mosaicml.com/blog/mpt-7b`, `2407.08319v1 → www.sdss5.org`
## Bug class B — relative figure/asset paths that 404
Images/assets are referenced by relative paths (often prefixed with the paper id) that do not resolve, so readers see broken figures:
| Witness article | Missing asset (example) |
|---|---|
| [2510.27684](https://arxiv.org/html/2510.27684) | `2510.27684v3/figures/images/20250921_subinvertal/reverse_nested.jpg` |
| [1502.00043](https://arxiv.org/html/1502.00043) | `assets/3mprices.jpeg` |
| [2207.05109v1](https://arxiv.org/html/2207.05109v1) | `x1.png` |
| [2210.11025v2](https://arxiv.org/html/2210.11025v2) | `figs/er1_shaw.eps` (EPS figure) |
## Worst offenders in the sample (by number of distinct broken `/html/` links)
`2510.15518v6` (67), `2510.27684` (63), `2606.29772v1` (38), `2607.25359v1` (28), `2607.20733v1` (24), `2602.22091v1` (22), `2607.26014v1` (20), `2607.25879v1` (19), `2607.19787v1` (18), `2607.11511v1` (18).
## Suggested fixes
- When emitting anchor hrefs, ensure external references carry a scheme: DOIs → `https://doi.org/…`, emails → `mailto:…`, bare `www.`/host URLs → `https://…`. (Class A above shows the correct form is already used *sometimes* for DOIs, so this looks like an inconsistency in reference/link handling.)
- Ensure figure/image `src` paths resolve for the served `/html/` (or `/html//`) base — the paper-id-prefixed relative paths in Class B do not resolve.
## Methodology / caveats
Derived from arxiv-browse origin request logs, HTTP 404s on `/html/*`, window 2026-08-05 19:00–21:00 UTC. Counts (975 links / 310 papers) are from that origin-visible sample only; crawler-amplified edge traffic for these URLs is substantially higher. Witness articles in Class A were confirmed by re-fetching the live HTML and locating the malformed `href`.
Contributor guide
Research direction
Start by tracing the HTML generation paths for anchor hrefs and figure/image src values, then reproduce the malformed links using the witness articles in the issue. Done means external references resolve with the correct schemes and figure paths load successfully under the served /html/ URLs without generating these 404s.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html
- Domain
- web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100