homepage Welcome to WebmasterWorld Guest from 54.197.40.130
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconHow to block tags?
  hundreds of undesired links !
3rashe18 4:33 pm on Sep 4, 2007 (gmt 0)
newpost thread iconBlocked through robots.txt . will the page be crawled ever
 
5getxb 6:02 am on Aug 31, 2007 (gmt 0)
newpost thread iconURI Only Listing
  Need a refresher course on this...
9pageoneresults 12:20 pm on Aug 24, 2007 (gmt 0)
newpost thread iconrobot.txt Security
  Can be read publically
4kkonline 10:08 am on Aug 24, 2007 (gmt 0)
newpost thread iconrobots.txt="noindex,follow"?
 
2davidcamus 9:55 am on Aug 24, 2007 (gmt 0)
newpost thread iconhow to block a page
  on a particular location
7experienced 9:47 am on Aug 24, 2007 (gmt 0)
newpost thread iconMultiple "Sitemap: " lines in robots.txt?
  Several TLDs use same site code / robots.txt.
2geoffyp 1:33 pm on Aug 10, 2007 (gmt 0)
newpost thread iconBaiduspider and multiple user-agent text file.
 
4Vimes 12:04 pm on Aug 7, 2007 (gmt 0)
newpost thread iconHow can you Disallow a Sub-Domain?
 
2zammo 10:11 am on Jul 31, 2007 (gmt 0)
newpost thread iconGoogle Exclude .swf Files
  How do i exclude swf files from indexing by google
5nxfx 5:26 pm on Jul 30, 2007 (gmt 0)
newpost thread iconHow to disallow entire content, except a particular location
 
2brakkar 12:07 pm on Jul 30, 2007 (gmt 0)
newpost thread icondisallowing url midway
 
2defanjos 5:20 pm on Jul 29, 2007 (gmt 0)
newpost thread iconBlock sub directory
 
5maxbear 5:24 am on Jul 28, 2007 (gmt 0)
newpost thread iconDoes this work?
  disallow multiple bots at once...
3Devilfear 6:48 am on Jul 27, 2007 (gmt 0)
newpost thread iconUsing multiple hash marks
  Does that conform to protocol?
5Reno 7:17 pm on Jul 25, 2007 (gmt 0)
newpost thread iconDisallowing Iframed pages
 
2Ozz13 1:29 pm on Jul 23, 2007 (gmt 0)
newpost thread iconProper format for disallow of URL
 
4Ozz13 1:14 pm on Jul 23, 2007 (gmt 0)
newpost thread iconWhat the dilly?
 
3tonynoriega 5:39 pm on Jul 18, 2007 (gmt 0)
newpost thread iconRe-instate pages previously blocked
  sorry for this noob question
4dibbern2 11:23 pm on Jul 16, 2007 (gmt 0)
newpost thread iconBroken Links on my web site
  I have too many broken links
2erdsah88 12:47 pm on Jul 16, 2007 (gmt 0)
newpost thread icon"Disallow: //" - error or meaningful?
  robots.txt crawler error
5sitetruth 11:58 pm on Jul 11, 2007 (gmt 0)
newpost thread iconBLOX dynamic urls with robots.txt
  BLOX dynamic urls with robots.txt
2bo0oost 11:50 am on Jul 11, 2007 (gmt 0)
newpost thread iconBlock affiliate tracking
  but keep the main page in Google
5chubba 2:26 pm on Jul 9, 2007 (gmt 0)
newpost thread iconWe can put sitempp.html to robots.txt?
  need some advise
2etgsgroup 12:00 pm on Jul 6, 2007 (gmt 0)
newpost thread iconA question about blocking IP from Nigerians
 
2rjr686 10:56 pm on Jul 3, 2007 (gmt 0)
Displaying Page 27 of 71<< Back 1 ... 17 18 19 20 21 22 23 24 25 26 [ 27 ] 28 29 30 31 32 33 34 35 36 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved