homepage Welcome to WebmasterWorld Guest from 54.205.119.163
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconGooglebot Wildcard
 
5payday 11:26 am on Nov 22, 2003 (gmt 0)
newpost thread iconImportance of Hotbot
 
5slobizman 9:08 am on Nov 20, 2003 (gmt 0)
newpost thread iconHow long until robots.txt works?
 
11slobizman 3:23 pm on Nov 18, 2003 (gmt 0)
newpost thread iconUnknown robots
  crawl, spider, and robot.
2synergy 4:10 am on Nov 16, 2003 (gmt 0)
newpost thread iconWould crawlers crawl this?
  Usiing the correct robots.txt technique
4ulysee 3:02 pm on Nov 15, 2003 (gmt 0)
newpost thread iconinternetseer
  billing for requested data
14plumsauce 7:47 pm on Nov 14, 2003 (gmt 0)
newpost thread iconPros and cons of having
  a robots.txt file?
3ideavirus 5:32 pm on Nov 13, 2003 (gmt 0)
newpost thread iconDoes anyone know what this is?
  I am thinking virus infected machine but unsure
3Crowbar 4:28 pm on Nov 12, 2003 (gmt 0)
newpost thread iconBlocking .log file with robots.txt
  Is it necessary, and syntax...?
7Robert_Charlton 6:59 am on Nov 11, 2003 (gmt 0)
newpost thread iconHow to make a page *inaccessible* to searches
  without using password protection?
12halfandhalf 8:04 pm on Nov 10, 2003 (gmt 0)
newpost thread iconthe wrong way to do robots.txt
  kinda funny, whitehouse.gov robots file
13amznVibe 7:42 pm on Nov 10, 2003 (gmt 0)
newpost thread iconWhy bother
  Is it worth blocking spambots with robots.txt
4WyrdoManx 11:52 pm on Nov 8, 2003 (gmt 0)
newpost thread iconDoes content type matter?
  is text/html okay
2kevinsl 8:27 am on Nov 8, 2003 (gmt 0)
newpost thread iconIE "Synchronize Offline while idle" is killing me
  Synchronize Offline
3Helice 12:46 am on Nov 6, 2003 (gmt 0)
newpost thread iconBots ignoring my robots.txt
  In place for a week, validates, but no use
12roddy 7:10 am on Nov 5, 2003 (gmt 0)
newpost thread iconBlocking a spider from a directory
 
3JVB_Mktg 4:30 am on Nov 5, 2003 (gmt 0)
newpost thread iconRobots.txt
  To use... or not to use? That is the question
6panic 1:29 am on Nov 5, 2003 (gmt 0)
newpost thread iconRobots.txt Validator
  We're sorry, this robots.txt does NOT validate.
2midge 12:48 pm on Nov 3, 2003 (gmt 0)
newpost thread iconRobot.txt - What is it, and why use it
 
4laffe 4:36 am on Nov 2, 2003 (gmt 0)
newpost thread iconrobots.txt syntex
  How to block a directory.
6jdancing 7:52 pm on Oct 31, 2003 (gmt 0)
newpost thread iconRobots.txt during development
  Anyone advise on this?
2stuntdubl 3:30 am on Oct 30, 2003 (gmt 0)
newpost thread iconbad Bots
 
3cgj5204 4:40 am on Oct 29, 2003 (gmt 0)
newpost thread iconRestrict Specific Domain Names from Being Indexed
  Using the robots.txt file on Google to restrict certain domains...
3jmh1970 1:36 am on Oct 29, 2003 (gmt 0)
newpost thread iconNew Forum : Robots.txt
 
1Brett_Tabke 9:24 pm on Oct 28, 2003 (gmt 0)
newpost thread iconPut your robots.txt on a diet
  Three easy ways to reduce bloat in robots.txt
20jdMorgan 8:36 pm on Oct 28, 2003 (gmt 0)
Displaying Page 64 of 71<< Back 1 ... 54 55 56 57 58 59 60 61 62 63 [ 64 ] 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved