Welcome to WebmasterWorld Guest from 54.157.222.62

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Serverel Corp. bot?
Suspicious probe
1 grandma_genie 3:55 am Aug 24, 2012
The Idaho crawlers...
I wonder who's behind this
12 bakedjake 7:24 pm Aug 23, 2012
Opera (Ukraine Edition)
Odd Substring In User Agent
4 iamzippy 9:36 pm Aug 21, 2012
Yandex.ru now running yandex.com spider
Wanted to let others know they may want to update their black/white list.[2] ( 1 2 )
46 JAB_Creations 11:58 pm Aug 20, 2012
British Library scraper
bl.uk_lddc_bot/3.1.1
6 dstiles 7:01 pm Aug 16, 2012
Bots by Hosting Company
6 incrediBILL 2:51 am Aug 16, 2012
SEOENGWorldBot
3 wilderness 12:43 am Aug 16, 2012
news bot
prateh.ru
8 keyplyr 8:04 pm Aug 13, 2012
google and favicon
strange google behavior
7 dupres01 11:40 am Aug 13, 2012
bottlenose.com
5 keyplyr 7:49 pm Aug 12, 2012
Ralocobot
1 keyplyr 8:01 pm Aug 10, 2012
Something from Myspace IP
5 motorhaven 7:27 pm Aug 9, 2012
Blekko/Scoutjet
10 keyplyr 4:21 am Aug 4, 2012
opera-mini.net bot tagting /feed address how to redirect it
bot hits /feed address using hsot name opera-mini.net
9 SaleF 6:08 pm Aug 2, 2012
131.253.21.0 - 131.253.47.255 msnbots?
I got a hit from an msnbot within this range, is this normal?
3 MickeyRoush 1:47 pm Aug 2, 2012
Google Web Preview
13 wilderness 7:48 pm Jul 29, 2012
MRSPUTNIK bot net?
2 incrediBILL 7:30 pm Jul 29, 2012
Google-Site-Verification/1.0 IPs
6 incrediBILL 1:46 am Jul 28, 2012
Googlebot Crawling from China IPs
Make sure you allow 203.208.60.1-203.208.60.255
16 incrediBILL 8:16 pm Jul 27, 2012
Mbbf 0.1 Dwl
Wikimedia
5 keyplyr 10:51 pm Jul 25, 2012
Whmcsxtra dedicated servers
AlphaServer
6 keyplyr 10:42 pm Jul 25, 2012
googlebot drink the brown acid?
Gobbledygook "GETs" from google
15 dupres01 8:37 pm Jul 25, 2012
Xenu Link Sleuth
Should anonymous linkers be blocked?[2] ( 1 2 )
39 grandma_genie 1:24 am Jul 24, 2012
Something weird with Google IP ranges
3 motorhaven 11:22 pm Jul 23, 2012
odd UA
26 wilderness 9:46 am Jul 23, 2012