homepage Welcome to WebmasterWorld Guest from 54.161.240.10
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobots INclusions standard?
 
13mikomido 12:28 am on Sep 9, 2007 (gmt 0)
newpost thread iconblocking the hostsite but not the hosted sites
 
3messemaeker 12:12 pm on Sep 7, 2007 (gmt 0)
newpost thread iconBlocking Windows Live robot/spider.
 
2Tidal2 10:56 am on Sep 7, 2007 (gmt 0)
newpost thread iconDo we have to use robots.txt on every website?
  for robots.txt issues
5masood82 8:42 am on Sep 6, 2007 (gmt 0)
newpost thread iconHow to block tags?
  hundreds of undesired links !
3rashe18 4:33 pm on Sep 4, 2007 (gmt 0)
newpost thread iconBlocked through robots.txt . will the page be crawled ever
 
5getxb 6:02 am on Aug 31, 2007 (gmt 0)
newpost thread iconURI Only Listing
  Need a refresher course on this...
9pageoneresults 12:20 pm on Aug 24, 2007 (gmt 0)
newpost thread iconrobot.txt Security
  Can be read publically
4kkonline 10:08 am on Aug 24, 2007 (gmt 0)
newpost thread iconrobots.txt="noindex,follow"?
 
2davidcamus 9:55 am on Aug 24, 2007 (gmt 0)
newpost thread iconhow to block a page
  on a particular location
7experienced 9:47 am on Aug 24, 2007 (gmt 0)
newpost thread iconMultiple "Sitemap: " lines in robots.txt?
  Several TLDs use same site code / robots.txt.
2geoffyp 1:33 pm on Aug 10, 2007 (gmt 0)
newpost thread iconBaiduspider and multiple user-agent text file.
 
4Vimes 12:04 pm on Aug 7, 2007 (gmt 0)
newpost thread iconHow can you Disallow a Sub-Domain?
 
2zammo 10:11 am on Jul 31, 2007 (gmt 0)
newpost thread iconGoogle Exclude .swf Files
  How do i exclude swf files from indexing by google
5nxfx 5:26 pm on Jul 30, 2007 (gmt 0)
newpost thread iconHow to disallow entire content, except a particular location
 
2brakkar 12:07 pm on Jul 30, 2007 (gmt 0)
newpost thread icondisallowing url midway
 
2defanjos 5:20 pm on Jul 29, 2007 (gmt 0)
newpost thread iconBlock sub directory
 
5maxbear 5:24 am on Jul 28, 2007 (gmt 0)
newpost thread iconDoes this work?
  disallow multiple bots at once...
3Devilfear 6:48 am on Jul 27, 2007 (gmt 0)
newpost thread iconUsing multiple hash marks
  Does that conform to protocol?
5Reno 7:17 pm on Jul 25, 2007 (gmt 0)
newpost thread iconDisallowing Iframed pages
 
2Ozz13 1:29 pm on Jul 23, 2007 (gmt 0)
newpost thread iconProper format for disallow of URL
 
4Ozz13 1:14 pm on Jul 23, 2007 (gmt 0)
newpost thread iconWhat the dilly?
 
3tonynoriega 5:39 pm on Jul 18, 2007 (gmt 0)
newpost thread iconRe-instate pages previously blocked
  sorry for this noob question
4dibbern2 11:23 pm on Jul 16, 2007 (gmt 0)
newpost thread iconBroken Links on my web site
  I have too many broken links
2erdsah88 12:47 pm on Jul 16, 2007 (gmt 0)
newpost thread icon"Disallow: //" - error or meaningful?
  robots.txt crawler error
5sitetruth 11:58 pm on Jul 11, 2007 (gmt 0)
Displaying Page 27 of 71<< Back 1 ... 17 18 19 20 21 22 23 24 25 26 [ 27 ] 28 29 30 31 32 33 34 35 36 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved