homepage Welcome to WebmasterWorld Guest from 54.227.40.166
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconUser agent in robots.txt and browscap.ini...
  Why is there a format different?
6Pushycat 11:39 pm on May 30, 2002 (gmt 0)
newpost thread iconrobots.txt -- block one user-agent, allow all others
 
2luma 1:44 am on May 23, 2002 (gmt 0)
newpost thread iconRobot.txt file help
  How to create this file
7beasscr 6:30 pm on May 16, 2002 (gmt 0)
newpost thread iconrobots.txt
  Generates a 404 error in my logs
9zeus 1:52 pm on May 3, 2002 (gmt 0)
newpost thread iconDoes robots.txt override meta tag?
 
8kira 7:12 pm on May 2, 2002 (gmt 0)
newpost thread iconRobots.txt validator updated.
 
7Brett_Tabke 6:48 pm on Apr 19, 2002 (gmt 0)
newpost thread iconWhat is robots.txt syntax
 
22Alec_Doggone 9:07 am on Apr 19, 2002 (gmt 0)
newpost thread iconSubmitting robots.txt to google
  Would google look sooner for it?
17Yidaki 8:19 am on Apr 19, 2002 (gmt 0)
newpost thread iconRobots.txt Pattern Matching
  How Well Does It Work?
5WebGuerrilla 1:15 pm on Mar 15, 2002 (gmt 0)
newpost thread iconRobots.txt Question?
  How often does Ink check for the file??
6MarkHutch 5:11 pm on Mar 11, 2002 (gmt 0)
newpost thread iconDo I need a robots.txt file in my directory?
  Google Bot - Whats Up with it?
10jady 9:58 pm on Feb 24, 2002 (gmt 0)
newpost thread iconHow important is the Robots.txt file now?
  ....in 2002
29ScottM 1:16 am on Jan 28, 2002 (gmt 0)
newpost thread iconRobots.txt Question
 
7mud 3:21 am on Jan 17, 2002 (gmt 0)
newpost thread iconI am writing my 1st robots.txt :) - never bothered before
  Anyone know any crawlers I should add ??
3diddlydazz 4:11 am on Jan 12, 2002 (gmt 0)
newpost thread iconMSIECrawler not obeying my robots.txt :(
  any ideas ?
5diddlydazz 3:09 am on Jan 12, 2002 (gmt 0)
newpost thread iconRobots.txt standard
 
4count_zer0 11:21 am on Dec 7, 2001 (gmt 0)
newpost thread iconrobots.txt
 
7Kenny B 1:02 pm on Sep 25, 2001 (gmt 0)
newpost thread iconRobots.txt specific query
 
8IanTurner 7:19 pm on Sep 24, 2001 (gmt 0)
newpost thread iconbasic robots.txt question
 
5kapow 4:31 pm on Aug 2, 2001 (gmt 0)
newpost thread iconI am new to all of this about Spiders...Robot.txt
  Help Message
8lightwriter 12:42 am on Jul 23, 2001 (gmt 0)
newpost thread iconrobots.txt
  Banning dodgy spiders - do they take any notice?
4themoff 7:35 pm on Jul 13, 2001 (gmt 0)
newpost thread iconRobots.txt
  Does Google honour it....
12Rusky 5:58 am on Jul 5, 2001 (gmt 0)
newpost thread iconFear and Loathing of Robots.txt
  Coming Clean - Confessions of a a robo'phobic
12Brett_Tabke 1:23 am on Jun 25, 2001 (gmt 0)
newpost thread iconBad style in robots.txt file. Why?
 
5billyblu 10:25 am on Jun 1, 2001 (gmt 0)
newpost thread iconRobots.txt Validator
 
4Froggyman 9:17 am on May 26, 2001 (gmt 0)
Displaying Page 70 of 71<< Back 1 ... 60 61 62 63 64 65 66 67 68 69 [ 70 ] 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved