homepage Welcome to WebmasterWorld Guest from 54.145.191.14
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread icongoogle allow - disallow all others
 
6jimmy19 7:20 am on Jul 30, 2004 (gmt 0)
newpost thread iconMSN and Yahoo both missing robots.txt
 
4bppilot 3:26 pm on Jul 29, 2004 (gmt 0)
newpost thread iconYahoo robots.txt problem
  what redirect?
4yowza 9:53 pm on Jul 27, 2004 (gmt 0)
newpost thread iconGoogle listing disallowed pages
  this normal?
2sonik 6:44 pm on Jul 22, 2004 (gmt 0)
newpost thread iconrobots.txt for different language sites
  can use charset tag to allow/disallow?
4a2ztranslate 11:51 pm on Jul 20, 2004 (gmt 0)
newpost thread iconabout subdomain and robots
  should i copy robots to all subdomains?
2webon 3:33 am on Jul 18, 2004 (gmt 0)
newpost thread iconrobots.txt question
 
3motorcycle 5:30 pm on Jul 17, 2004 (gmt 0)
newpost thread iconDisallowing Robot's Effect on Sub-Domain
  Will it disallow the sub domains as well
2bppilot 5:53 am on Jul 14, 2004 (gmt 0)
newpost thread iconWhy is Google following my exclusions?
  Robots.txt file says dont go and Google does!
5anxvariety 5:49 am on Jul 14, 2004 (gmt 0)
newpost thread iconWhy are my .swf files still being indexed by Googlebot?
 
3ichthyous 6:26 pm on Jul 13, 2004 (gmt 0)
newpost thread iconexcluding dynamic pages in all search engines
  excluding dynamic pages in robots.txt
3toddmc04 2:49 pm on Jul 13, 2004 (gmt 0)
newpost thread iconGeneral Guide to Robots.txt
  is there one here :)
4rj87uk 7:51 pm on Jul 7, 2004 (gmt 0)
newpost thread icondomain.com vs www.domain.com
  different sites
9HitProf 4:01 pm on Jul 7, 2004 (gmt 0)
newpost thread iconDo i need the txt file?
  if i want to allow everyone
4markbiz 12:59 am on Jul 5, 2004 (gmt 0)
newpost thread iconI'm not understanding the different makes of bots
  Can I stop some, while allowing others?
2tomequin 10:11 pm on Jul 1, 2004 (gmt 0)
newpost thread iconYahoo! Slurp - robots.txt
  Not obeyng robots.txt
5EarWig 1:59 pm on Jun 29, 2004 (gmt 0)
newpost thread iconMaking robots?
 
4lifedeuce 1:22 pm on Jun 29, 2004 (gmt 0)
newpost thread iconDisallow pages in a subdirectory
  Basic help please
3pjuk 8:40 pm on Jun 24, 2004 (gmt 0)
newpost thread iconsite setup for spider?
 
5Gemelli_no1 12:51 pm on Jun 23, 2004 (gmt 0)
newpost thread iconneeded?
  is it completely neccisary
3natty 1:34 pm on Jun 15, 2004 (gmt 0)
newpost thread iconRobots dot text for all user files?
  robots.txt all public_html
4kj6loh 7:21 pm on Jun 11, 2004 (gmt 0)
newpost thread iconwildcards in the Disallow field,
  non standard?
3pmac 4:19 pm on Jun 11, 2004 (gmt 0)
newpost thread iconwhere to place robots.txt?
  robots newbie alert
6Duke_of_Url 10:17 pm on Jun 10, 2004 (gmt 0)
newpost thread iconHow should I use a robots.txt file on a virtual account?
  I want to ban spiders from one site only
3soccer_star 2:01 pm on Jun 10, 2004 (gmt 0)
newpost thread iconRobots.txt for yahoo and google.com
  google and yahoo robots.txt
3adisakmai 4:47 am on Jun 10, 2004 (gmt 0)
Displaying Page 56 of 72<< Back 1 ... 46 47 48 49 50 51 52 53 54 55 [ 56 ] 57 58 59 60 61 62 63 64 65 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved