homepage Welcome to WebmasterWorld Guest from 54.211.14.159
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconhttp code 200 mystery
  is the page served or not!?
3dailyclicker 5:14 pm on Oct 13, 2007 (gmt 0)
newpost thread iconDo robots get cookies?
  robots cookies possible dumb question
4Michel_Samuel 10:55 am on Oct 11, 2007 (gmt 0)
newpost thread iconmultiple disallows
  excluding multiple root directories.
5Michel_Samuel 10:05 pm on Oct 10, 2007 (gmt 0)
newpost thread iconMultiple robots.txt files in different directories -will it work?
  placing a robots.txt file in sub directories.
3Michel_Samuel 7:44 pm on Oct 9, 2007 (gmt 0)
newpost thread iconKeep physical hosting domain from being indexed?
 
5php4U 2:21 am on Oct 9, 2007 (gmt 0)
newpost thread iconHow do I not allow google
 
2pizzaguy 10:31 pm on Oct 3, 2007 (gmt 0)
newpost thread iconShould this file be "Public"
  Is it good practice to have the robots.txt file.......
4Propools 11:58 am on Sep 26, 2007 (gmt 0)
newpost thread iconProper URL & File Structure
  Fighting with Robots. Txt
3srkmodi 12:34 pm on Sep 24, 2007 (gmt 0)
newpost thread iconUse & or & amp; in robots.txt
  Which to use?
3g1smd 1:53 pm on Sep 21, 2007 (gmt 0)
newpost thread iconUsing allow to index search-only pages
 
3timmbbo 12:31 pm on Sep 18, 2007 (gmt 0)
newpost thread iconDisallow - wait a day - add pages
 
1brevetoxin 1:36 am on Sep 12, 2007 (gmt 0)
newpost thread iconRobots.txt to improve ranks
  Will blocking of internal pages through robots.txt improve SE rankings
3pro_seo 5:18 am on Sep 10, 2007 (gmt 0)
newpost thread iconRobots INclusions standard?
 
13mikomido 12:28 am on Sep 9, 2007 (gmt 0)
newpost thread iconblocking the hostsite but not the hosted sites
 
3messemaeker 12:12 pm on Sep 7, 2007 (gmt 0)
newpost thread iconBlocking Windows Live robot/spider.
 
2Tidal2 10:56 am on Sep 7, 2007 (gmt 0)
newpost thread iconDo we have to use robots.txt on every website?
  for robots.txt issues
5masood82 8:42 am on Sep 6, 2007 (gmt 0)
newpost thread iconHow to block tags?
  hundreds of undesired links !
3rashe18 4:33 pm on Sep 4, 2007 (gmt 0)
newpost thread iconBlocked through robots.txt . will the page be crawled ever
 
5getxb 6:02 am on Aug 31, 2007 (gmt 0)
newpost thread iconURI Only Listing
  Need a refresher course on this...
9pageoneresults 12:20 pm on Aug 24, 2007 (gmt 0)
newpost thread iconrobot.txt Security
  Can be read publically
4kkonline 10:08 am on Aug 24, 2007 (gmt 0)
newpost thread iconrobots.txt="noindex,follow"?
 
2davidcamus 9:55 am on Aug 24, 2007 (gmt 0)
newpost thread iconhow to block a page
  on a particular location
7experienced 9:47 am on Aug 24, 2007 (gmt 0)
newpost thread iconMultiple "Sitemap: " lines in robots.txt?
  Several TLDs use same site code / robots.txt.
2geoffyp 1:33 pm on Aug 10, 2007 (gmt 0)
newpost thread iconBaiduspider and multiple user-agent text file.
 
4Vimes 12:04 pm on Aug 7, 2007 (gmt 0)
newpost thread iconHow can you Disallow a Sub-Domain?
 
2zammo 10:11 am on Jul 31, 2007 (gmt 0)
Displaying Page 27 of 72<< Back 1 ... 17 18 19 20 21 22 23 24 25 26 [ 27 ] 28 29 30 31 32 33 34 35 36 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved