homepage Welcome to WebmasterWorld Guest from 54.196.62.23
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobots.txt for Wordpress and Vbulletin together - Help
  Robots.txt for Wordpress and Vbulletin - Help
7anticipation 11:17 pm on Sep 3, 2011 (gmt 0)
newpost thread iconGoogle's Search Result Display Pattern
 
5webexpert2u 3:42 am on Aug 31, 2011 (gmt 0)
newpost thread iconrobots.txt disallow searching reoccuring folders
 
6jchristi 2:14 am on Aug 30, 2011 (gmt 0)
newpost thread iconhow to disallow feed in robot.txt readable by all bots
 
5bhavana 4:42 am on Aug 17, 2011 (gmt 0)
newpost thread iconSitemap for out of stock products
 
2sam222 6:43 am on Aug 9, 2011 (gmt 0)
newpost thread iconwhat we do for our robots
 
6lucy24 2:22 am on Aug 7, 2011 (gmt 0)
newpost thread iconQuestion on Sitelinks
  sitelinks
3sledge81 10:26 pm on Aug 2, 2011 (gmt 0)
newpost thread iconHow can I block these kinds of urls?
 
3starchild 1:45 am on Aug 2, 2011 (gmt 0)
newpost thread iconsitemap for mobile versions
 
2sam222 2:11 am on Jul 30, 2011 (gmt 0)
newpost thread iconFree tools for creating a sitemap
 
11Fotiman 12:13 pm on Jul 27, 2011 (gmt 0)
newpost thread iconBlocking Spiders
  How to block spiders from crawling your site
6Brian0275 9:15 pm on Jul 25, 2011 (gmt 0)
newpost thread iconwhich part of "disallow" did you not understand?
 
9lucy24 2:46 am on Jul 25, 2011 (gmt 0)
newpost thread iconwww, www21, wwww, etc. Displaying Same Content & Indexed By SEs
  What to do?
5getxb 6:48 am on Jul 22, 2011 (gmt 0)
newpost thread iconHow to manage this in robots.txt
  disallow a folder but include one page
10pankajj 1:36 pm on Jul 21, 2011 (gmt 0)
newpost thread iconDisallowed Googlebot-Image bot still spidering
 
7cyberdyne 8:47 am on Jul 9, 2011 (gmt 0)
newpost thread iconArchive.org
  for sites not crawled since 2006
8Neil_McRae 6:04 pm on Jul 1, 2011 (gmt 0)
newpost thread iconGooglebot not respecting custom crawl rate
 
13rowan194 3:38 am on Jun 22, 2011 (gmt 0)
newpost thread iconsitemap for images on subdomain
 
4AlexB77 6:50 am on Jun 11, 2011 (gmt 0)
newpost thread iconWhat's the difference between bing bot and Microsoft?
  Microsoft IP staying too long...banned for now, what's with that?
3erlandc 10:21 pm on Jun 4, 2011 (gmt 0)
newpost thread iconGoogle, Bing and Yahoo agree on Microformats?
  Was that discussed here: schema.org?
5pontifex 11:00 pm on Jun 3, 2011 (gmt 0)
newpost thread iconrobots.txt wildcard syntax?
  is this syntax ok?
5maha 9:39 pm on May 31, 2011 (gmt 0)
newpost thread iconIndex Page Removal, Serious Problem Please Help
  robots.txt or whats the issue? non-technical person here
6faust1 10:25 am on May 29, 2011 (gmt 0)
newpost thread iconGenerating a huge sitemap on shared host
  Creating a sitemap with thousands of URL'son a shared hosting plan.
1BrainDed 5:54 pm on May 23, 2011 (gmt 0)
newpost thread iconrobots.txt - disallow any url containing.
  using robots.txt and dynamic urls for phpbb3 forum
7BrainDed 10:28 pm on May 18, 2011 (gmt 0)
newpost thread iconWhy Indexing count Fluctuates?
 
2ihavegotyou 11:35 pm on May 17, 2011 (gmt 0)
Displaying Page 7 of 72<< Back 1 2 3 4 5 6 [ 7 ] 8 9 10 11 12 13 14 15 16 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved