homepage Welcome to WebmasterWorld Guest from 54.204.66.38
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconA list of american search engines.
  search engine list. American.
9Michel_Samuel 12:49 am on Jun 22, 2007 (gmt 0)
newpost thread iconDoes this trailing slash mean anything?
  <meta name="robots" content="noindex" />
11bouncybunny 7:10 pm on Jun 20, 2007 (gmt 0)
newpost thread iconNeed help with a specific robots.txt question
 
4Kurgano 9:18 pm on Jun 12, 2007 (gmt 0)
newpost thread iconabout:blank >>>what is this
  then the page loads?
4twtnyc 11:44 pm on Jun 8, 2007 (gmt 0)
newpost thread iconAllow rather than Disallow
  Is it possible?
5Marshall 10:30 pm on Jun 7, 2007 (gmt 0)
newpost thread iconWhy is msnbot ignoring this robot's.txt rule?
  Wildcard disallow ignored
3bouncybunny 12:17 pm on Jun 6, 2007 (gmt 0)
newpost thread iconBlock all Nutch variants. Possible?
 
6morags 9:02 pm on Jun 5, 2007 (gmt 0)
newpost thread iconUse FP 2000 and Want to Keep Goo from Indexing It
  How?
7ember 12:19 pm on May 31, 2007 (gmt 0)
newpost thread iconAllow google to crawl on only primary domains
  How to achieve exclusion on multiple domains using obj.conf & robots.txt
3justlonging1 12:35 pm on May 30, 2007 (gmt 0)
newpost thread iconblock all except in robots.txt
 
2visualscope 2:26 pm on May 28, 2007 (gmt 0)
newpost thread iconWhat is a bot trap?
 
2fischermx 4:28 pm on May 26, 2007 (gmt 0)
newpost thread iconCustom rule for google, url args and wordpress feeds
 
6Kurt 2:28 pm on May 24, 2007 (gmt 0)
newpost thread iconWordpress - Robots.txt to fix Duplicate Content
  I need a robots.txt file to prevent dup content from being crawled
7jman123 2:16 pm on May 23, 2007 (gmt 0)
newpost thread iconAlexa robot
  issue with alexa bot & alexa toolbar users
6Michel_Samuel 10:45 am on May 20, 2007 (gmt 0)
newpost thread iconRobots.txt
  Can you exclude a parameter in robots.txt?
3North_Shore 9:22 pm on May 18, 2007 (gmt 0)
newpost thread iconrobots.txt mess-up
 
4strawbleu 9:12 pm on May 18, 2007 (gmt 0)
newpost thread iconXML Stemap
 
6ajitaryan 11:35 am on May 16, 2007 (gmt 0)
newpost thread iconIs this valid?
 
3aha7 12:34 pm on May 9, 2007 (gmt 0)
newpost thread iconDisallowing content previously indexed
  What happens to indexed content subsequently blocked by robots.txt
3daveozzz 11:22 am on May 9, 2007 (gmt 0)
newpost thread iconduplicate content indexing
 
11coburn 6:34 pm on May 7, 2007 (gmt 0)
newpost thread iconDenying one page with multiple variables
  page.html?item=1, page.html?item=2, page.html?item=3
7wfernley 6:21 pm on May 7, 2007 (gmt 0)
newpost thread iconCan anyone post a complete robots.txt file here?
 
4PowerUp 2:29 pm on May 7, 2007 (gmt 0)
newpost thread iconHow do I disallow bots from my Wordpress year Archive Folders
  I guess this is a common question but I can't find an answer
4Drum 5:26 pm on May 5, 2007 (gmt 0)
newpost thread iconAdd Sitemap To robots.txt For Autodiscovery[2] ( 1 2 )
  Ask.com, Google, Microsoft Live Search and Yahoo!
32OldWolf 7:56 am on May 4, 2007 (gmt 0)
newpost thread iconHow is alexa getting past my stoppers
  stop alexa index
1Michel_Samuel 3:31 pm on May 2, 2007 (gmt 0)
Displaying Page 28 of 71<< Back 1 ... 18 19 20 21 22 23 24 25 26 27 [ 28 ] 29 30 31 32 33 34 35 36 37 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved