homepage Welcome to WebmasterWorld Guest from 54.198.157.6
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconWeird Code Return from Googlebot when robots.txt was spidered
 
35x54u 9:14 pm on Nov 25, 2003 (gmt 0)
newpost thread iconProblem with robots.txt
 
4Nisse 8:08 am on Nov 25, 2003 (gmt 0)
newpost thread iconrobots and dynamic url
  How to ban robots from a dynamic url
3Ledfish 8:09 pm on Nov 23, 2003 (gmt 0)
newpost thread iconGooglebot Wildcard
 
5payday 11:26 am on Nov 22, 2003 (gmt 0)
newpost thread iconImportance of Hotbot
 
5slobizman 9:08 am on Nov 20, 2003 (gmt 0)
newpost thread iconHow long until robots.txt works?
 
11slobizman 3:23 pm on Nov 18, 2003 (gmt 0)
newpost thread iconUnknown robots
  crawl, spider, and robot.
2synergy 4:10 am on Nov 16, 2003 (gmt 0)
newpost thread iconWould crawlers crawl this?
  Usiing the correct robots.txt technique
4ulysee 3:02 pm on Nov 15, 2003 (gmt 0)
newpost thread iconinternetseer
  billing for requested data
14plumsauce 7:47 pm on Nov 14, 2003 (gmt 0)
newpost thread iconPros and cons of having
  a robots.txt file?
3ideavirus 5:32 pm on Nov 13, 2003 (gmt 0)
newpost thread iconDoes anyone know what this is?
  I am thinking virus infected machine but unsure
3Crowbar 4:28 pm on Nov 12, 2003 (gmt 0)
newpost thread iconBlocking .log file with robots.txt
  Is it necessary, and syntax...?
7Robert_Charlton 6:59 am on Nov 11, 2003 (gmt 0)
newpost thread iconHow to make a page *inaccessible* to searches
  without using password protection?
12halfandhalf 8:04 pm on Nov 10, 2003 (gmt 0)
newpost thread iconthe wrong way to do robots.txt
  kinda funny, whitehouse.gov robots file
13amznVibe 7:42 pm on Nov 10, 2003 (gmt 0)
newpost thread iconWhy bother
  Is it worth blocking spambots with robots.txt
4WyrdoManx 11:52 pm on Nov 8, 2003 (gmt 0)
newpost thread iconDoes content type matter?
  is text/html okay
2kevinsl 8:27 am on Nov 8, 2003 (gmt 0)
newpost thread iconIE "Synchronize Offline while idle" is killing me
  Synchronize Offline
3Helice 12:46 am on Nov 6, 2003 (gmt 0)
newpost thread iconBots ignoring my robots.txt
  In place for a week, validates, but no use
12roddy 7:10 am on Nov 5, 2003 (gmt 0)
newpost thread iconBlocking a spider from a directory
 
3JVB_Mktg 4:30 am on Nov 5, 2003 (gmt 0)
newpost thread iconRobots.txt
  To use... or not to use? That is the question
6panic 1:29 am on Nov 5, 2003 (gmt 0)
newpost thread iconRobots.txt Validator
  We're sorry, this robots.txt does NOT validate.
2midge 12:48 pm on Nov 3, 2003 (gmt 0)
newpost thread iconRobot.txt - What is it, and why use it
 
4laffe 4:36 am on Nov 2, 2003 (gmt 0)
newpost thread iconrobots.txt syntex
  How to block a directory.
6jdancing 7:52 pm on Oct 31, 2003 (gmt 0)
newpost thread iconRobots.txt during development
  Anyone advise on this?
2stuntdubl 3:30 am on Oct 30, 2003 (gmt 0)
newpost thread iconbad Bots
 
3cgj5204 4:40 am on Oct 29, 2003 (gmt 0)
Displaying Page 64 of 71<< Back 1 ... 54 55 56 57 58 59 60 61 62 63 [ 64 ] 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved