homepage Welcome to WebmasterWorld Guest from 50.19.172.0
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread icondisallow all : including host
 
4moua 3:57 pm on Aug 22, 2005 (gmt 0)
newpost thread iconGoogle Sitemap file name?
  What should I call my Google Sitemap?
2jetnovo 3:21 pm on Aug 22, 2005 (gmt 0)
newpost thread iconSpiders and redirects to disallowed directories
  Are the disallowed pages indexed?
2cigjonser 7:18 pm on Aug 19, 2005 (gmt 0)
newpost thread iconallow some and exclude the rest in robots.txt
 
9jcmiras 5:24 am on Aug 19, 2005 (gmt 0)
newpost thread iconcan you have 2 robots.txt files on site?
  one in main and one in subdomain
8stiksandstones 12:10 am on Aug 19, 2005 (gmt 0)
newpost thread iconWhat bots ignore robots.txt?
 
8Jon12345 10:56 pm on Aug 18, 2005 (gmt 0)
newpost thread iconValidator Questions
  Need Help Deciphering Validator Output
7Jioffri 6:07 am on Aug 18, 2005 (gmt 0)
newpost thread iconQuick question on robots.txt
  Can you separate the agent and the allows/disallows?
2cmendla 11:19 pm on Aug 17, 2005 (gmt 0)
newpost thread iconGoogle's Nonstandard robots.txt is Not Valid
  Googles robots.txt Does not validate
6forak 1:59 pm on Aug 17, 2005 (gmt 0)
newpost thread iconSearch engines picking up our backend?
  Developement site picked up
3financialhost 12:46 pm on Aug 11, 2005 (gmt 0)
newpost thread iconStill no PR for site
 
3gnuman 8:39 pm on Aug 10, 2005 (gmt 0)
newpost thread iconCan excluding with one word also exclude phrases?
 
3Marcia 10:47 pm on Aug 7, 2005 (gmt 0)
newpost thread iconNew algo about Robots.txt
  Is there any new algo introduced regarding robots.txt?
4Graham_Hans 2:31 am on Aug 7, 2005 (gmt 0)
newpost thread iconDmoz robots.txt
  Is the dmoz.org/world disallowed?
4angiolo 8:05 am on Aug 6, 2005 (gmt 0)
newpost thread iconHow to exclude Oracle Portal Help pages using robots.txt
  Need to exclude Oracle Portal Help Pages from search engines
10atul_iiit 9:12 am on Aug 5, 2005 (gmt 0)
newpost thread iconhttp://<hostname.domainname>/robots.txt is access denied can robots
  Can robots read robots.txt if I get access denied for this file
4atul_iiit 4:25 pm on Aug 4, 2005 (gmt 0)
newpost thread iconWhat's Robot.txt?
  What's the importance of Robot.txt?
3Lowkei 9:02 am on Jul 31, 2005 (gmt 0)
newpost thread iconDon't want to index dynamic pages
  But would it be a mistake to block them?
5chopin2256 2:12 am on Jul 30, 2005 (gmt 0)
newpost thread iconCould 744 permissions on robots.txt cause problems
  all other files are set to 644
5dbar 7:41 pm on Jul 29, 2005 (gmt 0)
newpost thread iconRobots.txt viewable
  viewing the robots.txt file
2eleap 3:27 pm on Jul 27, 2005 (gmt 0)
newpost thread iconHow to exclude a few pages?
 
2jack38 10:25 am on Jul 27, 2005 (gmt 0)
newpost thread iconBlocking a file extension with robots.txt
  Inherited a site with a problem
5abbeyvet 5:45 pm on Jul 26, 2005 (gmt 0)
newpost thread iconHow to avoid getting print-pages indexed?
  What's the best way for Google?
1simczak 3:42 pm on Jul 24, 2005 (gmt 0)
newpost thread iconRobots.txt stops automated link software
 
2jdhuk 11:01 pm on Jul 18, 2005 (gmt 0)
newpost thread iconWhich editor?
  Notepad or editor in UNIX
3Bali_Freak 12:15 pm on Jul 18, 2005 (gmt 0)
Displaying Page 45 of 72<< Back 1 ... 35 36 37 38 39 40 41 42 43 44 [ 45 ] 46 47 48 49 50 51 52 53 54 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved