matomo-org / matomo-org/device-detector
Detect bots from HTTP_FROM
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 3.5k
- Forks
- 513
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 11
Description
I think this should be implemented, and I have three reasons for this:
- User agents change and it's hard to keep up will all bots, worldwide, especially if they don't get reported so we can add them, and also we can detect them right away.
- Bing uses
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0andHTTP_FROMisbingbot(at)microsoft.com. - Bots are required to send
HTTP_FROMhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/From
If HTTP_FROM is detected then it should be treated as generic bot, then further check for specific bot name.
-
"<?=print(9347655345-4954366);?>"
Mozilliqa"<?=print(9347655345-4954366);?>" -
TGVnaXRpbWF0ZSBsaW5rIHRyYWNrZXI=
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0 -
bingbot(at)microsoft.com
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0 -
crawler@alexa.com
ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) -
crawler@rois.ac.jp
Mozilla/5.0 (compatible; Cotoyogi/4.0; +https://ds.rois.ac.jp/center8/crawler/ -
googlebot(at)googlebot.com
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) -
gptbot(at)openai.com
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.0; +https://openai.com/gptbot) -
morten@cs.umn.edu
SuggestBot/1.0 -
oai-searchbot@openai.com
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot -
pigafetta-bot(at)visual-seo.com
Mozilla 5.0 (compatible; Pigafetta/0.5; +http://visual-seo.com/Pigafetta-Bot) -
robot@seokicks.de
Mozilla/5.0 (compatible; SEOkicks; +https://www.seokicks.de/robot.html) -
root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.oast.site
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.oast.site -
support@search.yandex.ru
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) -
tech@babbar.tech
Mozilla/5.0 (compatible; Barkrowler/0.9; +https://babbar.tech/crawler) -
the.knowledge.ai@gmail.com
The Knowledge AI -
wc@verisign.com
Mozilla/5.0 (compatible; ips-agent)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or code entry points are named. Start by tracing the existing bot-detection path and how request headers and user-agent strings are handled, then determine where HTTP_FROM should trigger generic bot detection and where the listed addresses should identify specific bots. Done means the proposed cases are covered without relying only on changing user-agent strings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100