homepage Welcome to WebmasterWorld Guest from 184.73.52.98
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconWhy is google indexing this banned directory?
 
5ichthyous 4:55 pm on Jan 26, 2007 (gmt 0)
newpost thread iconhttps on google and robots.txt
 
2concepts99 1:54 pm on Jan 24, 2007 (gmt 0)
newpost thread icondirectory structure question
 
5union_jack 7:43 pm on Jan 22, 2007 (gmt 0)
newpost thread iconblock a subdirectory
 
2concepts99 2:05 pm on Jan 22, 2007 (gmt 0)
newpost thread iconGoogle not respecting robots.txt
  Found >500 pages in Google index that are forbiden by robots.txt
5roots 5:54 am on Jan 22, 2007 (gmt 0)
newpost thread icondomo arigato
  mr. roboto
3ricnut 11:52 pm on Jan 20, 2007 (gmt 0)
newpost thread iconDisallow Slurp from all files except two
  Yahoo likes only one page, so I'd like to just 'give' it that.
2tigertom 3:00 am on Jan 20, 2007 (gmt 0)
newpost thread iconYahoo Slurp Bot misbehaving
  .. or is it someone masquerading?
4tigertom 9:27 am on Jan 17, 2007 (gmt 0)
newpost thread iconBlocking Site in some countries
 
3eldinboy 5:55 pm on Jan 15, 2007 (gmt 0)
newpost thread iconWhat to do with duplicate pages
 
3olba 2:12 am on Jan 12, 2007 (gmt 0)
newpost thread iconWhy should I have a robots.txt file?
  Isn't it OK if I don't?
23beren 8:45 am on Jan 11, 2007 (gmt 0)
newpost thread iconGoogle Site Diagnostics - robots.txt blocks urls
 
4akreider 7:44 pm on Jan 5, 2007 (gmt 0)
newpost thread iconRobots txt to solve gallery duplicates
  can someone confirm this is correct
3caines 10:54 am on Jan 5, 2007 (gmt 0)
newpost thread iconHow many search engine understand crawl delay?
 
1foxfox 3:38 pm on Jan 4, 2007 (gmt 0)
newpost thread iconMultiple Domains Issue
 
3kdobson99 4:58 am on Jan 4, 2007 (gmt 0)
newpost thread iconDo spiders follow links from pages with the noindex tag?
  As opposed to links from robots files?
4bouncybunny 6:59 am on Jan 2, 2007 (gmt 0)
newpost thread iconDirecting Spiders to specific Servers
 
2TheSeoGuy 1:50 pm on Dec 29, 2006 (gmt 0)
newpost thread iconI had robots.txt blocking my phpbb folders which are now deleted
  Should I remove that line from robots.txt now?
2cmendla 4:16 pm on Dec 24, 2006 (gmt 0)
newpost thread iconDisallowing Indexed files in Google
 
5leo11877 2:01 pm on Dec 23, 2006 (gmt 0)
newpost thread iconDisallowing by date.
  Is it possible to keep some page and disallow new ones...?
3bouncybunny 8:31 am on Dec 20, 2006 (gmt 0)
newpost thread iconWorried about profile links
 
4Asia_Expat 3:53 pm on Dec 19, 2006 (gmt 0)
newpost thread iconIs it possible to exclude a URL by specifying an anchor?
  Excluding #...?
3bouncybunny 12:40 am on Dec 17, 2006 (gmt 0)
newpost thread iconRemoving Pages - Does Robots.txt Help?
 
3akreider 4:17 am on Dec 16, 2006 (gmt 0)
newpost thread iconAllowing only some pages for ia archive
 
5sosoo 8:20 am on Dec 13, 2006 (gmt 0)
newpost thread iconMediapartners-Google fellow the robots.txt?
 
2AlphaWu 1:41 pm on Dec 12, 2006 (gmt 0)
Displaying Page 31 of 71<< Back 1 ... 21 22 23 24 25 26 27 28 29 30 [ 31 ] 32 33 34 35 36 37 38 39 40 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved