Welcome to WebmasterWorld Guest from 107.20.34.173

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Google Translate
7 wilderness 2:43 am Dec 11, 2013
YaBrowser
15 lucy24 10:04 am Dec 10, 2013
wscheck
1 lucy24 1:17 am Dec 10, 2013
this time it's Sogou
61.135 returns
1 lucy24 11:32 pm Dec 9, 2013
New stupid referer spammer
Referer spam via agent string (mistyped)
9 yaimapitu 2:58 pm Dec 8, 2013
rarely used
12 lucy24 9:22 am Dec 1, 2013
New Scraping Tool
9 keyplyr 9:07 am Nov 28, 2013
Bixocrawler
Another data miner
1 dstiles 4:02 pm Nov 24, 2013
Cleaning House
blocked IP ranges may have changed
1 keyplyr 10:19 am Nov 24, 2013
Yahoo phone home
19 lucy24 8:07 pm Nov 23, 2013
Amazon aws, dedicatedcpanel, serverloft Issues
These domains are dedicatedly screwing my site - Need Help.
10 Future 7:59 pm Nov 23, 2013
TECH UPDATE: Bots as Browsers Using JavaScript
The Bot Detection Game Has Changed[2] ( 1 2 )
48 incrediBILL 9:18 pm Nov 11, 2013
google-xrawler
Unknown User agent "google-xrawler"
15 DutchTheWiz 7:51 pm Nov 7, 2013
Zylon
4 lucy24 8:32 pm Nov 6, 2013
Is CompSpyBot really an NSASpyBot?
17 evsiz 7:55 pm Oct 30, 2013
50.16.178.xx Amazon of course
linkCheck
2 Bewenched 8:21 pm Oct 24, 2013
Who/what is ns4006554.ip-198-27-80.net?
7 iomfan 2:01 am Oct 22, 2013
another one for the profilers
6 lucy24 6:52 am Oct 19, 2013
What is GABot/1.0
6 woodpecker 11:33 pm Oct 12, 2013
"DAPPER-HOST-IP" why Baidu ?
"DAPPER-HOST-IP" usrer agent baidu
18 aboshakeeb 9:20 am Oct 10, 2013
CrawlDaddy abot Site Harvester
3 WarmGlow 11:37 pm Sep 28, 2013
meanpath
10 lucy24 6:42 am Sep 21, 2013
International Data Centers
6 incrediBILL 7:31 pm Sep 16, 2013
Dot Exe in User Agent String
10 lucy24 9:55 pm Sep 10, 2013
same old same old YisouSpider
Yisou back from the dead?
4 lucy24 1:23 am Sep 6, 2013

Featured Threads

Hot Threads This Week

Hot Threads This Month