homepage Welcome to WebmasterWorld Guest from 54.204.73.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt confusion
  question about where to put robots.txt
10SplitPersona 3:13 pm on Mar 17, 2005 (gmt 0)
newpost thread iconexclude vb with robots
 
1johndk 9:55 am on Mar 16, 2005 (gmt 0)
newpost thread iconFilenames have spaces - How to reflect in robots.txt
  Website has filenames that contain spaces. What to do?
4SanDiego_Art 2:54 am on Mar 16, 2005 (gmt 0)
newpost thread iconrestriction for a certain country
  is it possible to do that?
3lucky 8:06 pm on Mar 15, 2005 (gmt 0)
newpost thread iconneed help blocking a site
  not sure if this is possible
11willeffects 5:23 am on Mar 15, 2005 (gmt 0)
newpost thread iconPriceGrabber UserAgent
 
1driwashsolutions 1:59 am on Mar 15, 2005 (gmt 0)
newpost thread iconcrawl confusion
  Understanding my hits log
5cardpro 8:59 am on Mar 14, 2005 (gmt 0)
newpost thread iconi see the bigger site have the bots not crawling them
  i see the bigger site have the bots not crawling them
2indodpendo 7:12 pm on Mar 13, 2005 (gmt 0)
newpost thread iconim using this sites robits.txt file but my main page gets psa's
  im using this sites robits.txt file but my main page gets psa's
2indodpendo 7:10 pm on Mar 13, 2005 (gmt 0)
newpost thread iconPHP site
  Disallowing pages
3dj_lysuc 12:30 pm on Mar 13, 2005 (gmt 0)
newpost thread iconis it necessary for dynamic pages
  aspx pages
3qais77 2:38 pm on Mar 12, 2005 (gmt 0)
newpost thread iconOnly Allowing Googlebot, Msnbot and Slurp
  is it possible?
4globay 4:28 pm on Mar 11, 2005 (gmt 0)
newpost thread icondisallow all from subdirectory
 
11Driven 1:26 pm on Mar 10, 2005 (gmt 0)
newpost thread iconRobots.txt will it be ignored
 
1azzbacwordz 6:39 pm on Mar 9, 2005 (gmt 0)
newpost thread iconrobots.txt
  I need a robots.txt
4indodpendo 1:35 am on Mar 9, 2005 (gmt 0)
newpost thread iconsimple question concerning adsense
 
1esllou 10:26 am on Mar 7, 2005 (gmt 0)
newpost thread iconNeed help with robot.txt for .php forums
  robot.txt, .php, forums,
2Freakindj 2:44 pm on Mar 5, 2005 (gmt 0)
newpost thread iconBot blocking versus IP throttling for bad bots
  Dealing with rippers and their ilk
1Webwork 4:10 am on Mar 3, 2005 (gmt 0)
newpost thread iconDoes this cancle out the robots.txt
  Does this cancle out the robots.txt
3indodpendo 2:16 pm on Mar 1, 2005 (gmt 0)
newpost thread iconMultiple Directories
  robot.txt, subdirectories, protected directories
1Ansuz 1:47 pm on Feb 28, 2005 (gmt 0)
newpost thread iconRobot terms defined
  The differences and similarities between nofollow, noindex and robots.txt
1u_to_the_sixth 7:29 am on Feb 26, 2005 (gmt 0)
newpost thread iconMaximum size for robots.txt
  Does any one know what it is?
6blend27 9:40 pm on Feb 25, 2005 (gmt 0)
newpost thread iconrobots.txt
  robots.txt
4indodpendo 8:59 pm on Feb 25, 2005 (gmt 0)
newpost thread iconHow can I get my whole site out of search engines and Google cache?
 
2mig1234 5:19 pm on Feb 23, 2005 (gmt 0)
newpost thread iconBanning Affiliate Links from being Indexed
 
1bears5122 11:57 pm on Feb 22, 2005 (gmt 0)
Displaying Page 50 of 72<< Back 1 ... 40 41 42 43 44 45 46 47 48 49 [ 50 ] 51 52 53 54 55 56 57 58 59 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved