Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Instapaper/7.7.1.2 CFNetwork/978.0.7 Darwin/18.6.0
3 tangor 3:37 am Aug 3, 2019
newspaper
9 Pfui 10:32 pm Jul 29, 2019
Facebook pretending to be Bing
Bing UA from Facebook IP
8 dstiles 10:21 am Jul 29, 2019
?source=post page---------------------------
What the--?
6 Pfui 6:24 am Jul 29, 2019
Chrome Lighthouse redux
3 lucy24 12:13 am Jul 29, 2019
Safe to Block HeadlessChrome?
9 glakes 1:16 am Jul 26, 2019
two from Facebook
13 lucy24 5:39 pm Jul 25, 2019
Another funny fake user agent string
5 notriddle 5:48 pm Jul 21, 2019
Hit from "Blockchain Network Solutions"
Not sure what it's trying to accomplish
3 SumGuy 10:06 am Jun 4, 2019
Hits from 104.132.0.0/16 (Google) - human or bot?
8 SumGuy 2:04 pm Jun 3, 2019
Apache OpenOffice
10 lucy24 7:23 pm May 31, 2019
FaceBook iPhone UA
2 wilderness 1:33 pm May 31, 2019
OrgProbe/2.0.0
Checking to see if my site is up?
2 SumGuy 9:20 pm May 20, 2019
Hits from Cisco Ironport
What is their purpose?
2 SumGuy 2:52 pm May 20, 2019
Getting hammered by Bingbot
7 tangor 12:50 pm May 18, 2019
Evergreen Googlebot Now Runs Latest Chromium Rendering Engine
6 engine 12:37 am May 15, 2019
possible new bing crawl range
2 lucy24 1:54 am Apr 12, 2019
Bots start looking for back.zip, backup.zip, etc.
Get rid of those if you have them in root.
3 SumGuy 12:32 pm Apr 10, 2019
Google-speakr
6 phranque 9:23 pm Apr 5, 2019
User Agent is Microsoft Office something or other
5 SumGuy 8:29 pm Apr 4, 2019
Google 34
2 lucy24 1:48 am Apr 2, 2019
Facebook cortex
8 lucy24 1:21 am Apr 2, 2019
35.244.0.0/14 - Google or Merit (U-Mich) ?
4 SumGuy 6:23 pm Apr 1, 2019
Getting lots of hits from this Cloudsigma. what's it?
3 born2run 5:58 am Mar 24, 2019
Apple applebot increasing activity
Increasing hits to interior web pages
4 SumGuy 4:32 am Mar 18, 2019