homepage Welcome to WebmasterWorld Guest from 54.197.92.233
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt vs htaccess for control
  if the robots.txt is not dependable why not just use htaccess?
6ccubed99 3:39 am on Jan 28, 2006 (gmt 0)
newpost thread iconrobots.txt and htaccess
 
4agradation 11:17 pm on Jan 26, 2006 (gmt 0)
newpost thread iconTypical robots.txt?
  Anybody got a same robots.txt?
11Lit 6:58 pm on Jan 26, 2006 (gmt 0)
newpost thread iconDisallow for sub-domain directory?
 
7palmpal 4:18 pm on Jan 26, 2006 (gmt 0)
newpost thread iconBlocking old pages
 
1Nutter 3:21 pm on Jan 26, 2006 (gmt 0)
newpost thread iconWildcards special syntax for Google
  publish Google unswer about special Google syntax in robots.txt plus examle
2spiral 3:44 am on Jan 26, 2006 (gmt 0)
newpost thread iconSpiders finding you against all odds
 
3Lyndsay 6:49 pm on Jan 23, 2006 (gmt 0)
newpost thread iconyahoo/slurp spidering
  We don't want these pages indexed by yahoo
9latimer 8:28 am on Jan 23, 2006 (gmt 0)
newpost thread iconCorrect way to ban a directory
 
5palmpal 3:18 am on Jan 23, 2006 (gmt 0)
newpost thread iconWWW vs. Non WWW
  Site indexed twice need robot.txt help
3carjocky 4:10 pm on Jan 20, 2006 (gmt 0)
newpost thread iconUnknowingly restricting some directories
 
2estaquieto 4:40 am on Jan 20, 2006 (gmt 0)
newpost thread iconMSNBOT and robots.txt
 
2Impala 4:31 am on Jan 20, 2006 (gmt 0)
newpost thread iconHow-to: robots.txt white-listing with PHP/Apache
 
2encyclo 2:00 pm on Jan 16, 2006 (gmt 0)
newpost thread iconProhibiting folders by name regardless of root relation
  Is this possible or not?
6Jeremy_H 2:38 pm on Jan 15, 2006 (gmt 0)
newpost thread iconquestion about robots.txt, am I banning a directory by accident
 
17arbitrary 3:32 am on Jan 15, 2006 (gmt 0)
newpost thread iconQuick 'Sanity Check'
 
3markd 10:20 am on Jan 11, 2006 (gmt 0)
newpost thread icondirectory
 
4sylvieg 4:38 am on Jan 11, 2006 (gmt 0)
newpost thread iconRemoving pages from Google's cache
  Is it best to allow a "robots" "noarchive" crawl first?
1Phil_Payne 2:05 am on Jan 10, 2006 (gmt 0)
newpost thread iconDisallow ALL folders?
  but allow top level html
6Reno 9:14 pm on Jan 9, 2006 (gmt 0)
newpost thread iconHow to disallow widgets from being indexed
  Order of User-agents
2albatros 8:10 pm on Jan 9, 2006 (gmt 0)
newpost thread iconEmpty line after User-agent:
  Could this do any harm?
1taps 5:31 pm on Jan 7, 2006 (gmt 0)
newpost thread iconRobot Validator Problem
 
7bunky6767 1:13 pm on Jan 6, 2006 (gmt 0)
newpost thread iconBlocking all URLS to a website using the robots.txt file?
 
2scubajared 10:05 am on Jan 6, 2006 (gmt 0)
newpost thread iconTuring keys
 
2Ejap 5:57 pm on Jan 5, 2006 (gmt 0)
newpost thread iconABC of robots.txt
  Now that will be the best tutorial!
8Monzilla 2:19 pm on Jan 4, 2006 (gmt 0)
Displaying Page 40 of 72<< Back 1 ... 30 31 32 33 34 35 36 37 38 39 [ 40 ] 41 42 43 44 45 46 47 48 49 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved