ArchiveBox / ArchiveBox/ArchiveBox
Enhancement: Use the same URL layout as Archive.org for viewing ArchiveBox Snapshots `https://archive.org/web/<URL>`
- Dominant language
- Python
- Stars
- 28.4k
- Forks
- 1.6k
- Avg merge
- 7h 54m
- Merged PRs (30d)
- 21
Description
To visit an archived version of a website (or archive it automatically) on Archive.org, one can just visit `http://web.archive.org/web/https://example.com/` and it will redirect to `http://web.archive.org/web/20230116145642/https://example.com/` (or whatever the most recent snapshot timestamp is).
To really emobdy the tagline "ArchiveBox is a self-hosted version of archive.org" we should properly support their URL scheme too.
e.g.
- `https://demo.archivebox.io/web/https://example.com` should redirect to the most recent snapshot `https://demo.archivebox.io/web/20230116145642/https://example.com`
- note: support both the ArchiveBox-style timestamp in unix timestamp format e.g. `1673919713` or the Archive.org-style `20230116145642` format and truncated forms `2023`, `202301`, `20230116`
- note: also support visiting using snapshots using [ulid](https://github.com/ulid/spec) uuid instead of timestamp as slug, e.g. `https://demo.archivebox.io/01ARZ3NDEKTSV4RRFFQ69G5FAV/...`
- note: support auto prefix-matching slugs so that `2023` matches `202301`, `20230116`, `20230116145642` automatically, and `01AN4Z07BY` matches `01AN4Z07BY79KA1307SR9X4MV3 ` automatically
Full spec:
`https://demo.archivebox.io/web/` where `SLUG` can be:
- an original URL, with or without scheme, e.g. `https://example.com/index.html`, 'example.com/index.html' ➡️ redirect to most recent snapshot for `https://demo.archivebox.io/web/20230116145642/https://example.com/index.html`
- an ArchiveBox snapshot UUID in [`ulid/spec`](https://github.com/ArchiveBox/ArchiveBox/issues/74) format `01AN4Z07BY79KA1307SR9X4MV3/index.html` or timestamp prefix `01AN4Z07BY/index.html` ➡️ redirect to that exact snapshot `https://demo.archivebox.io/web/20230116145642/https://example.com/index.html`
- an ArchiveBox snapshot timestamp in `YYMMDDHHMMSS`, shortened forms like `YYYYMM`, or unix timestamp format e.g. `20230116145642/index.html` or `202301161456/index.html`, `202301/index.html`, `1673919713/index.html` ➡️ redirect to most recent snapshot matching that prefix `https://demo.archivebox.io/web/20230116145642/https://example.com/index.html`
Subtasks:
- [x] adds derived `ulid` field + migration to coalesce old uuid and timestamp fields into new ulid format (+asserts all snapshot timestamps are valid and are between 1900 and 2100 AD) (done in v0.8.5)
- [x] update admin and index UI to show ULID of old UUID4 `xxxx-xxxx-xxxxxxx` format, add ULID diagram in docs breaking it down into timestamp and randomness
- [x] create disambiguation page to show all the matching results for a given SLUG if it's the prefix for multiple possible snapshots
- [ ] reject Snapshot UUIDs being created that begin with `0`, `1`,`2`,`htt` to make prefix-matching faster and less error prone (avoids clashing with `199x*`/`20**` year, `1*` unix timestamp, `01*` ULIDs, or `http(s?)` URL slug prefixes)
- [ ] add docs examples on how to truly "self-host your own archive.org", add screenshot side-by-side of URL bar examples for visiting snapshots on Archive.org and demo.Archivebox.io
Contributor guide
Research direction
No files, tests, or entry points are named. Start by locating the snapshot URL-routing code and existing timestamp or ULID lookup behavior, then trace the redirect and disambiguation paths. Done means the documented /web/ forms resolve or redirect correctly, including prefix matches and ambiguous results, with the remaining documentation and validation subtasks addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100