homepage Welcome to WebmasterWorld Guest from 50.19.206.49
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconOpera (Ukraine Edition)
  Odd Substring In User Agent
4iamzippy 9:36 pm on Aug 21, 2012 (gmt 0)
newpost thread iconYandex.ru now running yandex.com spider[2] ( 1 2 )
  Wanted to let others know they may want to update their black/white list.
46JAB_Creations 11:58 pm on Aug 20, 2012 (gmt 0)
newpost thread iconBritish Library scraper
  bl.uk_lddc_bot/3.1.1
6dstiles 7:01 pm on Aug 16, 2012 (gmt 0)
newpost thread iconBots by Hosting Company
 
6incrediBILL 2:51 am on Aug 16, 2012 (gmt 0)
newpost thread iconSEOENGWorldBot
 
3wilderness 12:43 am on Aug 16, 2012 (gmt 0)
newpost thread iconnews bot
  prateh.ru
8keyplyr 8:04 pm on Aug 13, 2012 (gmt 0)
newpost thread icongoogle and favicon
  strange google behavior
7dupres01 11:40 am on Aug 13, 2012 (gmt 0)
newpost thread iconbottlenose.com
 
5keyplyr 7:49 pm on Aug 12, 2012 (gmt 0)
newpost thread iconRalocobot
 
1keyplyr 8:01 pm on Aug 10, 2012 (gmt 0)
newpost thread iconSomething from Myspace IP
 
5motorhaven 7:27 pm on Aug 9, 2012 (gmt 0)
newpost thread iconBlekko/Scoutjet
 
10keyplyr 4:21 am on Aug 4, 2012 (gmt 0)
newpost thread iconopera-mini.net bot tagting /feed address how to redirect it
  bot hits /feed address using hsot name opera-mini.net
9SaleF 6:08 pm on Aug 2, 2012 (gmt 0)
newpost thread icon131.253.21.0 - 131.253.47.255 msnbots?
  I got a hit from an msnbot within this range, is this normal?
3MickeyRoush 1:47 pm on Aug 2, 2012 (gmt 0)
newpost thread iconGoogle Web Preview
 
13wilderness 7:48 pm on Jul 29, 2012 (gmt 0)
newpost thread iconMRSPUTNIK bot net?
 
2incrediBILL 7:30 pm on Jul 29, 2012 (gmt 0)
newpost thread iconGoogle-Site-Verification/1.0 IPs
 
6incrediBILL 1:46 am on Jul 28, 2012 (gmt 0)
newpost thread iconGooglebot Crawling from China IPs
  Make sure you allow 203.208.60.1-203.208.60.255
16incrediBILL 8:16 pm on Jul 27, 2012 (gmt 0)
newpost thread iconMbbf 0.1 Dwl
  Wikimedia
5keyplyr 10:51 pm on Jul 25, 2012 (gmt 0)
newpost thread iconWhmcsxtra dedicated servers
  AlphaServer
6keyplyr 10:42 pm on Jul 25, 2012 (gmt 0)
newpost thread icongooglebot drink the brown acid?
  Gobbledygook "GETs" from google
15dupres01 8:37 pm on Jul 25, 2012 (gmt 0)
newpost thread iconXenu Link Sleuth[2] ( 1 2 )
  Should anonymous linkers be blocked?
39grandma_genie 1:24 am on Jul 24, 2012 (gmt 0)
newpost thread iconSomething weird with Google IP ranges
 
3motorhaven 11:22 pm on Jul 23, 2012 (gmt 0)
newpost thread iconodd UA
 
26wilderness 9:46 am on Jul 23, 2012 (gmt 0)
newpost thread iconStart.exe User Agent?
  from fdcservers.net
4incrediBILL 8:03 pm on Jul 22, 2012 (gmt 0)
newpost thread iconNerdByNature.Bot
 
8incrediBILL 1:06 am on Jul 22, 2012 (gmt 0)
Displaying Page 18 of 215<< Back 1 ... 8 9 10 11 12 13 14 15 16 17 [ 18 ] 19 20 21 22 23 24 25 26 27 ... 215 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved