homepage Welcome to WebmasterWorld Guest from 54.226.168.96
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Library
Forum: Search Engine Spider and User Agent Identification
Displaying Topics 41 - 80 (48 total) Sorted by: Date-Last-Post, Direction: reverse
#Subjectsort orderDate
41: new post   Updated Bot List
Oct 4, 2003
42: new post   New Fast/Overture Robot UA
Sept 7, 2003
43: new post   YahooSeeker/1.0
What is it?
Aug 23, 2003
44: new post   Blocking WebPage Monitoring Services
Most of these services are unwelcome bandwidth sponges. Lets call them out, identify them, and block them together.
Apr 9, 2003
45: new post   Bad Boy Bot of the Week
Pick a bot, any bot: This weeks fun one is an affiliate bot out walking and not respecting robots.txt.
Dec 6, 2002
46: new post   Mysterious User Agent and 404 Random Strings
"Altavista and DIIbot use suspicious request methods to test 404 errors."
June 11, 2002
47: new post   Search Engine and Spider IP Lists
Nice resource on the major operating spiders.
June 10, 2002
48: new post   How important is the Robots.txt file now?
"Am I missing out on a lot of traffic by not having a robots.txt file? Or is that a thing of the past?"
Jan 25, 2002
#Subjectsort orderDate
open in new window=open in new window

   
<< Back 1 2 Next >>
Home / Forums Index / Library
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved