homepage Welcome to WebmasterWorld Guest from 54.242.126.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread icondirectory and subdirectory access
  allowing access to subdirectory but not main directory
3Mr_Cool_Dude 6:11 pm on Jan 2, 2008 (gmt 0)
newpost thread iconrobots.txt question
  robots.txt examples
2jim_knopf 5:43 pm on Jan 2, 2008 (gmt 0)
newpost thread iconReference robots.txt file?
  Does anyone have one?
3mmiller 11:30 pm on Dec 25, 2007 (gmt 0)
newpost thread iconhow to manage cname in robot
  don't was to be indexed
4experienced 2:10 pm on Dec 21, 2007 (gmt 0)
newpost thread icondisallow vs ban in .htaccess
 
12joergnw10 11:10 am on Dec 16, 2007 (gmt 0)
newpost thread iconHow do I prevent the SEs from crawling any links to my site w/HTTPS
  I don't want the crawlers to index anything w/ HTTPS protocol
3ZydoSEO 8:37 pm on Dec 9, 2007 (gmt 0)
newpost thread iconyahoo google msn etc.
  other searches
1ComputerHermit 5:56 pm on Dec 7, 2007 (gmt 0)
newpost thread iconProposal for robots.txt To Have Greater Flexibility
 
12engine 4:58 pm on Nov 30, 2007 (gmt 0)
newpost thread iconset crawl rate permanently to faster
 
1agushalim 5:39 pm on Nov 28, 2007 (gmt 0)
newpost thread iconWildcards on robots.txt disallows
 
11MindTwist 5:59 pm on Nov 27, 2007 (gmt 0)
newpost thread iconSite title and description missing in search results
  I had made some changes in robots.txt file
2ecmedia 2:12 pm on Nov 27, 2007 (gmt 0)
newpost thread iconperfect robot text files
  good robot text files
2likwidholdings 12:57 am on Nov 27, 2007 (gmt 0)
newpost thread iconGoogle showing site despite blocking in robots.txt
  The page is not cached though
3ecmedia 4:28 pm on Nov 20, 2007 (gmt 0)
newpost thread iconDisallow all except adsense
 
6thegreatpretender 4:40 pm on Nov 19, 2007 (gmt 0)
newpost thread iconHow to avoid double indexing
 
3mavi 4:43 pm on Nov 13, 2007 (gmt 0)
newpost thread iconRobots.txt Clarification
 
5smartcard 10:44 am on Nov 13, 2007 (gmt 0)
newpost thread iconRobot from colo is on behalf of Google?!
  Colo ISP claims robot crawling my site is from 3rd party Google contractor
5yodokame 4:03 am on Nov 12, 2007 (gmt 0)
newpost thread iconrobots.txt file
  how it affects the page rank of any website
2monikagaba 6:21 pm on Nov 1, 2007 (gmt 0)
newpost thread iconExclude ALL bots but major search engines
  Trying to make the perfect robots.txt
5serpmaster 12:03 pm on Oct 25, 2007 (gmt 0)
newpost thread iconblocking all except root (index.php)
  would disallow ALL , allow index work?
1bkode 7:56 am on Oct 24, 2007 (gmt 0)
newpost thread iconhttp code 200 mystery
  is the page served or not!?
3dailyclicker 5:14 pm on Oct 13, 2007 (gmt 0)
newpost thread iconDo robots get cookies?
  robots cookies possible dumb question
4Michel_Samuel 10:55 am on Oct 11, 2007 (gmt 0)
newpost thread iconmultiple disallows
  excluding multiple root directories.
5Michel_Samuel 10:05 pm on Oct 10, 2007 (gmt 0)
newpost thread iconMultiple robots.txt files in different directories -will it work?
  placing a robots.txt file in sub directories.
3Michel_Samuel 7:44 pm on Oct 9, 2007 (gmt 0)
newpost thread iconKeep physical hosting domain from being indexed?
 
5php4U 2:21 am on Oct 9, 2007 (gmt 0)
Displaying Page 26 of 72<< Back 1 ... 16 17 18 19 20 21 22 23 24 25 [ 26 ] 27 28 29 30 31 32 33 34 35 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved