homepage Welcome to WebmasterWorld Guest from 184.72.72.182
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconHow can I block these kinds of urls?
 
3starchild 1:45 am on Aug 2, 2011 (gmt 0)
newpost thread iconsitemap for mobile versions
 
2sam222 2:11 am on Jul 30, 2011 (gmt 0)
newpost thread iconFree tools for creating a sitemap
 
11Fotiman 12:13 pm on Jul 27, 2011 (gmt 0)
newpost thread iconBlocking Spiders
  How to block spiders from crawling your site
6Brian0275 9:15 pm on Jul 25, 2011 (gmt 0)
newpost thread iconwhich part of "disallow" did you not understand?
 
9lucy24 2:46 am on Jul 25, 2011 (gmt 0)
newpost thread iconwww, www21, wwww, etc. Displaying Same Content & Indexed By SEs
  What to do?
5getxb 6:48 am on Jul 22, 2011 (gmt 0)
newpost thread iconHow to manage this in robots.txt
  disallow a folder but include one page
10pankajj 1:36 pm on Jul 21, 2011 (gmt 0)
newpost thread iconDisallowed Googlebot-Image bot still spidering
 
7cyberdyne 8:47 am on Jul 9, 2011 (gmt 0)
newpost thread iconArchive.org
  for sites not crawled since 2006
8Neil_McRae 6:04 pm on Jul 1, 2011 (gmt 0)
newpost thread iconGooglebot not respecting custom crawl rate
 
13rowan194 3:38 am on Jun 22, 2011 (gmt 0)
newpost thread iconsitemap for images on subdomain
 
4AlexB77 6:50 am on Jun 11, 2011 (gmt 0)
newpost thread iconWhat's the difference between bing bot and Microsoft?
  Microsoft IP staying too long...banned for now, what's with that?
3erlandc 10:21 pm on Jun 4, 2011 (gmt 0)
newpost thread iconGoogle, Bing and Yahoo agree on Microformats?
  Was that discussed here: schema.org?
5pontifex 11:00 pm on Jun 3, 2011 (gmt 0)
newpost thread iconrobots.txt wildcard syntax?
  is this syntax ok?
5maha 9:39 pm on May 31, 2011 (gmt 0)
newpost thread iconIndex Page Removal, Serious Problem Please Help
  robots.txt or whats the issue? non-technical person here
6faust1 10:25 am on May 29, 2011 (gmt 0)
newpost thread iconGenerating a huge sitemap on shared host
  Creating a sitemap with thousands of URL'son a shared hosting plan.
1BrainDed 5:54 pm on May 23, 2011 (gmt 0)
newpost thread iconrobots.txt - disallow any url containing.
  using robots.txt and dynamic urls for phpbb3 forum
7BrainDed 10:28 pm on May 18, 2011 (gmt 0)
newpost thread iconWhy Indexing count Fluctuates?
 
2ihavegotyou 11:35 pm on May 17, 2011 (gmt 0)
newpost thread iconMy SITEMAPS are getting indexed--pls help implementing this tag
 
12donna130 1:16 pm on May 14, 2011 (gmt 0)
newpost thread iconthe good, the bad and the above the law?
 
1lucy24 10:03 pm on May 12, 2011 (gmt 0)
newpost thread iconblocking dynamic parts of forum
 
8viggen 6:42 pm on May 10, 2011 (gmt 0)
newpost thread iconSky IP requesting all files alphabetically
 
3cyberdyne 3:42 pm on May 7, 2011 (gmt 0)
newpost thread iconallowing & disallowing (robots.txt)
 
8mike2010 11:38 pm on May 4, 2011 (gmt 0)
newpost thread iconGoogle, robots, disallow
 
3cyberdyne 6:01 pm on May 3, 2011 (gmt 0)
newpost thread iconWebsite not showing in any search engine
 
21jesstp 3:20 pm on Apr 17, 2011 (gmt 0)
Displaying Page 7 of 71<< Back 1 2 3 4 5 6 [ 7 ] 8 9 10 11 12 13 14 15 16 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved