homepage Welcome to WebmasterWorld Guest from 54.147.196.159
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt - subdomains and domain
  where to place
3Rose_1171 8:06 am on Feb 18, 2006 (gmt 0)
newpost thread iconHttp: vs Https:
  What are the best ways to stop crawlers?
3thaedge 7:54 am on Feb 18, 2006 (gmt 0)
newpost thread iconDoes robots.txt work per directory or per domain?
 
3zver0 6:40 pm on Feb 16, 2006 (gmt 0)
newpost thread iconDo bots confuse .htm and .html files in robots.txt?
 
2subgenius 6:08 pm on Feb 16, 2006 (gmt 0)
newpost thread iconrobots.txt Advertising Methods
  The new trend...
3pageoneresults 4:49 pm on Feb 16, 2006 (gmt 0)
newpost thread iconBanning all subdirectories in a directory without the files?
 
5zver0 1:28 pm on Feb 16, 2006 (gmt 0)
newpost thread iconrenewed site not being indexed - old pages in serp cache
  how to fix robots.txt mistake
4Ring 10:47 am on Feb 15, 2006 (gmt 0)
newpost thread iconWhy doesn't this validate?
  Help please
4Vimes 11:04 am on Feb 14, 2006 (gmt 0)
newpost thread iconTripAdvisorBot?
 
1jexx 9:02 pm on Feb 13, 2006 (gmt 0)
newpost thread iconhow to disallow shopping cart
  rank is dropping since we set it up
2Lorel 7:39 am on Feb 12, 2006 (gmt 0)
newpost thread iconRobots.txt help
  Ok, maybe I did not understand how it works
15followgreg 9:20 pm on Feb 10, 2006 (gmt 0)
newpost thread icon? and * in robots.txt
  What stands a questionmark in the robots.txt for?
2solandre 5:50 pm on Feb 9, 2006 (gmt 0)
newpost thread iconMeta tags and Robots
 
5Buster42 4:48 pm on Feb 9, 2006 (gmt 0)
newpost thread iconCharacter Limit
  Has anyone head of a character limit in robots.txt?
7bumpaw 8:39 am on Feb 9, 2006 (gmt 0)
newpost thread iconIs this Robots.txt OK?
  Please check this robot.txt file
9mzconstruction 11:06 pm on Feb 3, 2006 (gmt 0)
newpost thread iconrobots.txt and SSL
 
3AlpacaAnnie 8:24 am on Feb 3, 2006 (gmt 0)
newpost thread iconAre different User-agent entries needed for versions of similar robot
 
3Ceyx 2:29 am on Feb 1, 2006 (gmt 0)
newpost thread iconRobots list service?
  Looking for a service provider or subscription service
2xbase234 1:41 am on Feb 1, 2006 (gmt 0)
newpost thread iconHow to disallow URLs with parameters?
  "/page-name.php?" or "/page-name.php"
11iProgram 11:34 pm on Jan 31, 2006 (gmt 0)
newpost thread iconThis OK?
 
3TonysDesigns 10:13 pm on Jan 30, 2006 (gmt 0)
newpost thread iconblock robots from images
  robot images
2gb_dev 9:12 pm on Jan 30, 2006 (gmt 0)
newpost thread iconIf my website dont have robots.txt
  robots.txt
7rusmart 8:50 pm on Jan 30, 2006 (gmt 0)
newpost thread iconDuplicate Content in Robots.txt file
 
2palmpal 12:17 am on Jan 30, 2006 (gmt 0)
newpost thread iconTo allow some robots only
 
9GoldenHammer 3:14 pm on Jan 28, 2006 (gmt 0)
newpost thread iconrobots.txt vs htaccess for control
  if the robots.txt is not dependable why not just use htaccess?
6ccubed99 3:39 am on Jan 28, 2006 (gmt 0)
Displaying Page 39 of 72<< Back 1 ... 29 30 31 32 33 34 35 36 37 38 [ 39 ] 40 41 42 43 44 45 46 47 48 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved