Link rot report: 110 of 1,430 links in the README are dead, classified, with the 160 false positives kept out

Open
#166 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Domain
documentation

Research direction

Start with README.md and compare its URLs against the 110 dead-link entries in this issue, keeping the 160 filtered results out of scope. Review each listed link for removal or an explicitly noted replacement, then verify that the README no longer contains the confirmed dead URLs and that the surviving links remain intact.

Written by the indexing model from the issue text.

Description

Your FAQ says it plainly - "OSINT tools constantly stop being supported, stop working correctly, or disappear altogether" - so I probed every link in the README and classified what came back. Posting the whole thing rather than a handful, because a rot list is only useful if it is complete and if the false positives have already been removed.

Method, so you can dispute any line: every unique URL in README.md was fetched with a normal desktop User-Agent, following redirects. Anything that did not return a success or redirect had its hostname re-resolved over DNS-over-HTTPS (Google and Cloudflare) rather than through my local resolver, and if it still had an address, a TCP connection was attempted. The DoH step is not pedantry: a local resolver returns an empty answer for "this name does not exist", for "this name exists but has no address", and for "your network filters this name", and those are three different facts. I had two domains look dead this week that were not.

Counts of 1430 unique links:

state n meaning
ok 1156 fine
404 58 host is healthy, that path is gone - usually a one-line fix
gone 31 NXDOMAIN from both public resolvers - never coming back
no address 15 domain still registered, no A or AAAA record at all
refused 6 address resolves, TCP connection refused - host up, service gone
filtered 160 403/429/503 or a timeout with healthy DNS
servfail 4 the other end's DNS is broken, which is not the same as dead

The 160 filtered ones are deliberately not listed and should not be touched. A 403 or a timeout with the domain resolving normally describes my access, not the link. If I dumped those in here you would find the first false positive within a minute and reasonably stop reading.


Gone - NXDOMAIN from both resolvers (31)
Registered, but no A or AAAA record (15)
Connection refused - host up, service gone (6)
404 - host fine, path gone (58)
Ambiguous - the other end's DNS is failing (SERVFAIL), reported but not called dead (4)

Two patterns worth more than the individual lines. Twenty-one of the 404s are GitHub repositories that no longer exist, which is your FAQ's "deleted their Github profile" case showing up at scale. And the free-tier hosts are a cluster of their own: every herokuapp.com and most glitch.me entries are gone, because those free tiers were withdrawn rather than because anyone abandoned the tool.

One last thing, and I will flag my own interest in it rather than bury it. The only browser-inspection tool in the list, developers.whatismybrowser.com/useragents/parse/, is one of the dead ones; it still exists at explore.whatismybrowser.com/useragents/parse/. I build a tool in that area - https://liarjs.dev/ - which reads a browser in JavaScript and then asks the network layer what it actually saw, and reports every place the two disagree. It is relevant to an OSINT collection mainly for the researcher's own side: it tells you whether the browser or container you are investigating from is presenting a coherent story, which a single-surface checker cannot. Free, no sign-up. Add it or do not - the list above is the point of this issue and it stands on its own either way.

Scripts and raw results are mine to share if you want to re-run this yourself later.

Dominant language
HTML
Stars
8.8k
Forks
1k
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cipher387/osint_stuff_tool_collection

All issues in cipher387/osint_stuff_tool_collection

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.