homepage Welcome to WebmasterWorld Guest from 54.211.47.170
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconIs it harmful if google indexes my /feed/ directory?
  how to dissallow safely
3brokenbricks 2:42 pm on Sep 9, 2005 (gmt 0)
newpost thread iconrobots.txt wildcard
  Will this work
7gosman 1:59 pm on Sep 6, 2005 (gmt 0)
newpost thread iconRobots.txt and frontpage
 
14sunflower12 1:32 pm on Sep 4, 2005 (gmt 0)
newpost thread iconThe codes required to call the robots
 
2benifer 10:04 pm on Aug 31, 2005 (gmt 0)
newpost thread iconGoogle Hammering My site (1500+ hits already today)
  How to edit robots.txt to set delay
5wfernley 12:05 pm on Aug 31, 2005 (gmt 0)
newpost thread iconWhy do you need robots.txt?
 
5gordele 6:38 pm on Aug 30, 2005 (gmt 0)
newpost thread iconSpecifying pages for robots to crawl
  Worth it or not?
1sixdeep 6:14 pm on Aug 29, 2005 (gmt 0)
newpost thread iconIncluding specific robots necessary or not?
 
7silverbytes 8:27 am on Aug 29, 2005 (gmt 0)
newpost thread iconhow do i ban image robots
 
2spivei 11:04 pm on Aug 28, 2005 (gmt 0)
newpost thread iconbasic idea of robots.txt
 
8moi_britphile 10:38 am on Aug 28, 2005 (gmt 0)
newpost thread iconBanning AskJeeves Bot
  Need to ban askJeeves
2Marcus_Aurelius 8:31 pm on Aug 27, 2005 (gmt 0)
newpost thread iconproblem with robots.txt file
  problem with robots.txt file
5raybo75 5:36 pm on Aug 27, 2005 (gmt 0)
newpost thread iconHow Often Should a Spider Come
  What is the right amount of access a spider should have
4sirjeffer 11:00 pm on Aug 22, 2005 (gmt 0)
newpost thread iconBot cn exclude this directory?
 
2tintin74 10:42 pm on Aug 22, 2005 (gmt 0)
newpost thread icondisallow all : including host
 
4moua 3:57 pm on Aug 22, 2005 (gmt 0)
newpost thread iconGoogle Sitemap file name?
  What should I call my Google Sitemap?
2jetnovo 3:21 pm on Aug 22, 2005 (gmt 0)
newpost thread iconSpiders and redirects to disallowed directories
  Are the disallowed pages indexed?
2cigjonser 7:18 pm on Aug 19, 2005 (gmt 0)
newpost thread iconallow some and exclude the rest in robots.txt
 
9jcmiras 5:24 am on Aug 19, 2005 (gmt 0)
newpost thread iconcan you have 2 robots.txt files on site?
  one in main and one in subdomain
8stiksandstones 12:10 am on Aug 19, 2005 (gmt 0)
newpost thread iconWhat bots ignore robots.txt?
 
8Jon12345 10:56 pm on Aug 18, 2005 (gmt 0)
newpost thread iconValidator Questions
  Need Help Deciphering Validator Output
7Jioffri 6:07 am on Aug 18, 2005 (gmt 0)
newpost thread iconQuick question on robots.txt
  Can you separate the agent and the allows/disallows?
2cmendla 11:19 pm on Aug 17, 2005 (gmt 0)
newpost thread iconGoogle's Nonstandard robots.txt is Not Valid
  Googles robots.txt Does not validate
6forak 1:59 pm on Aug 17, 2005 (gmt 0)
newpost thread iconSearch engines picking up our backend?
  Developement site picked up
3financialhost 12:46 pm on Aug 11, 2005 (gmt 0)
newpost thread iconStill no PR for site
 
3gnuman 8:39 pm on Aug 10, 2005 (gmt 0)
Displaying Page 44 of 71<< Back 1 ... 34 35 36 37 38 39 40 41 42 43 [ 44 ] 45 46 47 48 49 50 51 52 53 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved