homepage Welcome to WebmasterWorld Guest from 50.17.66.61
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDisallow for sub-domain directory?
 
7palmpal 4:18 pm on Jan 26, 2006 (gmt 0)
newpost thread iconBlocking old pages
 
1Nutter 3:21 pm on Jan 26, 2006 (gmt 0)
newpost thread iconWildcards special syntax for Google
  publish Google unswer about special Google syntax in robots.txt plus examle
2spiral 3:44 am on Jan 26, 2006 (gmt 0)
newpost thread iconSpiders finding you against all odds
 
3Lyndsay 6:49 pm on Jan 23, 2006 (gmt 0)
newpost thread iconyahoo/slurp spidering
  We don't want these pages indexed by yahoo
9latimer 8:28 am on Jan 23, 2006 (gmt 0)
newpost thread iconCorrect way to ban a directory
 
5palmpal 3:18 am on Jan 23, 2006 (gmt 0)
newpost thread iconWWW vs. Non WWW
  Site indexed twice need robot.txt help
3carjocky 4:10 pm on Jan 20, 2006 (gmt 0)
newpost thread iconUnknowingly restricting some directories
 
2estaquieto 4:40 am on Jan 20, 2006 (gmt 0)
newpost thread iconMSNBOT and robots.txt
 
2Impala 4:31 am on Jan 20, 2006 (gmt 0)
newpost thread iconHow-to: robots.txt white-listing with PHP/Apache
 
2encyclo 2:00 pm on Jan 16, 2006 (gmt 0)
newpost thread iconProhibiting folders by name regardless of root relation
  Is this possible or not?
6Jeremy_H 2:38 pm on Jan 15, 2006 (gmt 0)
newpost thread iconquestion about robots.txt, am I banning a directory by accident
 
17arbitrary 3:32 am on Jan 15, 2006 (gmt 0)
newpost thread iconQuick 'Sanity Check'
 
3markd 10:20 am on Jan 11, 2006 (gmt 0)
newpost thread icondirectory
 
4sylvieg 4:38 am on Jan 11, 2006 (gmt 0)
newpost thread iconRemoving pages from Google's cache
  Is it best to allow a "robots" "noarchive" crawl first?
1Phil_Payne 2:05 am on Jan 10, 2006 (gmt 0)
newpost thread iconDisallow ALL folders?
  but allow top level html
6Reno 9:14 pm on Jan 9, 2006 (gmt 0)
newpost thread iconHow to disallow widgets from being indexed
  Order of User-agents
2albatros 8:10 pm on Jan 9, 2006 (gmt 0)
newpost thread iconEmpty line after User-agent:
  Could this do any harm?
1taps 5:31 pm on Jan 7, 2006 (gmt 0)
newpost thread iconRobot Validator Problem
 
7bunky6767 1:13 pm on Jan 6, 2006 (gmt 0)
newpost thread iconBlocking all URLS to a website using the robots.txt file?
 
2scubajared 10:05 am on Jan 6, 2006 (gmt 0)
newpost thread iconTuring keys
 
2Ejap 5:57 pm on Jan 5, 2006 (gmt 0)
newpost thread iconABC of robots.txt
  Now that will be the best tutorial!
8Monzilla 2:19 pm on Jan 4, 2006 (gmt 0)
newpost thread iconDefine allowable spider times[2] ( 1 2 )
  MSN is going nutz!
31Reno 1:44 pm on Jan 4, 2006 (gmt 0)
newpost thread iconNew empty site beeing spidered
 
2sren 3:08 am on Dec 29, 2005 (gmt 0)
newpost thread iconremove PHP from Google.com
  remove PHP from Google.com
2maimax 7:32 am on Dec 28, 2005 (gmt 0)
Displaying Page 40 of 72<< Back 1 ... 30 31 32 33 34 35 36 37 38 39 [ 40 ] 41 42 43 44 45 46 47 48 49 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved