homepage Welcome to WebmasterWorld Guest from 54.205.144.54
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification

Search Engine Spider and User Agent Identification

Forum Charter, Library, Moderated by: Ocean10000 & incrediBILL Forum Options: Reset Last Read Pointers

PreModerated forum to id search engine spiders

 
new Subject Messages Started by Last Message
newpost thread iconand another gets on.
 
5lucy24 1:25 pm on Jan 18, 2013 (gmt 0)
newpost thread iconLots of interesting user agents
 
4Bewenched 7:54 pm on Jan 15, 2013 (gmt 0)
newpost thread iconBaidu's IPs - Which Are Legit?
 
12incrediBILL 7:21 pm on Jan 13, 2013 (gmt 0)
newpost thread iconLSSRocketCrawler/1.0 LightspeedSystems
  69.84.207.246
3Bewenched 5:57 pm on Jan 13, 2013 (gmt 0)
newpost thread iconsitemap.xml requests
 
3wilderness 12:38 am on Jan 13, 2013 (gmt 0)
newpost thread iconTiscali Communicator (scraper)
  on SteepHost (Ukraine)
3not2easy 9:22 pm on Jan 12, 2013 (gmt 0)
newpost thread iconCould someone spare a set of Anti-AWS (and other offenders) CSF rules?
 
21script 8:45 am on Jan 12, 2013 (gmt 0)
newpost thread iconAre these considered as bad bots ?
 
23pkKumar 3:00 am on Jan 11, 2013 (gmt 0)
newpost thread iconmsn bot has gone crazy
 
15bwnbwn 10:52 pm on Jan 9, 2013 (gmt 0)
newpost thread iconApple
 
6wilderness 2:19 am on Jan 9, 2013 (gmt 0)
newpost thread iconbrowser versions
 
3wilderness 1:39 am on Jan 9, 2013 (gmt 0)
newpost thread iconNetscape
 
5wilderness 10:15 pm on Jan 8, 2013 (gmt 0)
newpost thread iconEntanet
  Entanet-adsl
2not2easy 10:05 pm on Jan 8, 2013 (gmt 0)
newpost thread iconUndefined Error in Logs[2] ( 1 2 )
 
39yaashul 5:28 am on Jan 7, 2013 (gmt 0)
newpost thread iconOne for the UA profilers
 
5Bewenched 6:38 am on Jan 6, 2013 (gmt 0)
newpost thread iconMicrosoft or not?
  70.37.130.8
9Bewenched 5:56 am on Jan 5, 2013 (gmt 0)
newpost thread iconYandex translate
 
4lucy24 6:36 pm on Jan 3, 2013 (gmt 0)
newpost thread iconthat Yahoo thingy
 
2wilderness 6:13 am on Dec 28, 2012 (gmt 0)
newpost thread iconAnyone know the name of Wayback Machine's robot?
  One site has not been crawled all year... wondering why.
19JeffOstroff 8:46 pm on Dec 27, 2012 (gmt 0)
newpost thread iconHuman or robot?
  another one for the profilers
4lucy24 9:09 am on Dec 27, 2012 (gmt 0)
newpost thread iconNing/1.0
  eat my 403
15Bewenched 2:51 am on Dec 27, 2012 (gmt 0)
newpost thread iconTosCrawler
 
14lucy24 6:55 pm on Dec 25, 2012 (gmt 0)
newpost thread iconWIndows NT 9.0 - Yeeeehaaa
  Back from the future!
6blend27 10:54 pm on Dec 23, 2012 (gmt 0)
newpost thread iconGoogle scraping through China?
 
7dstiles 8:13 pm on Dec 20, 2012 (gmt 0)
newpost thread iconData reseller bot shortlist?
 
5Sgt_Kickaxe 5:40 pm on Dec 14, 2012 (gmt 0)
Displaying Page 11 of 215<< Back 1 2 3 4 5 6 7 8 9 10 [ 11 ] 12 13 14 15 16 17 18 19 20 ... 215 Next >>
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved