homepage Welcome to WebmasterWorld Guest from 54.196.206.80
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconWhat in the world is a robot.txt?
  I need this to be stated simple for me
5Claudia_T 6:35 pm on Feb 10, 2004 (gmt 0)
newpost thread iconHelp! I need to disallow a specific query string
 
1Hissingsid 11:26 am on Feb 10, 2004 (gmt 0)
newpost thread iconPlease verify
 
2mikemcs 4:25 am on Feb 10, 2004 (gmt 0)
newpost thread iconGooglebot chasing .php tracking script
  is robots.txt enough?
2Edouard_H 12:51 am on Feb 9, 2004 (gmt 0)
newpost thread icon301 redirects & robots.txt
  What is the implication of a 301 redirect on robots.txt
2Jerry_Pannu 10:17 pm on Feb 7, 2004 (gmt 0)
newpost thread iconTesting robots.txt
  Why won't my site get spidered
2DavidAtWork 5:56 pm on Feb 5, 2004 (gmt 0)
newpost thread iconI've been looking at the WW robots.txt ...
  ...but why are these spiders disallowed?
7elgumbo 1:32 pm on Feb 4, 2004 (gmt 0)
newpost thread iconGoogleBot
  Will using robots exclusion and meta exclusion hurt me?
1mercury 9:12 pm on Feb 3, 2004 (gmt 0)
newpost thread iconSpiders that ignore or skip robots.txt
  Can bad spiders be identified?
4DavidAtWork 2:05 pm on Feb 3, 2004 (gmt 0)
newpost thread iconRobots Meta and .txt
  Where?
4admin_amoroustoys 1:34 pm on Feb 3, 2004 (gmt 0)
newpost thread iconrobots.txt used in court case
 
2transactiongeek 1:27 pm on Feb 3, 2004 (gmt 0)
newpost thread iconwhat is the "KAIST AITrc Crawler" bot?
  it ignored my "robots.txt" file and got caught in my bot trap
4stapel 12:01 pm on Feb 3, 2004 (gmt 0)
newpost thread iconIs it illegal to ignore a robots.txt?
  Could I sue google?
2transactiongeek 6:44 am on Feb 3, 2004 (gmt 0)
newpost thread iconrobots metatag
  I have noindex,follow but the page is indexed
2nevetS 4:41 am on Feb 3, 2004 (gmt 0)
newpost thread iconsave and upload robots.txt
 
4helenp 8:27 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this robots.txt blocking anybody
 
8nalavanje 2:20 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this enough
 
5dougie 11:20 am on Feb 2, 2004 (gmt 0)
newpost thread iconHow Do I Code My Robots.txt File To Disallow Subdomians?
  Is it Disallow: /apple.domain.com or Disallow: /apple/
2Christyl_Stevens 9:34 am on Feb 2, 2004 (gmt 0)
newpost thread iconasterias/2.0
  Looks for streaming media files...
6pendanticist 11:43 pm on Feb 1, 2004 (gmt 0)
newpost thread iconwill all robots crawl the sites without robot.txt?
  robot crawl, robot.txt
2johnlim 3:08 am on Feb 1, 2004 (gmt 0)
newpost thread iconWhat will Inktomi do in this scenario?
 
18flobaby 6:02 am on Jan 31, 2004 (gmt 0)
newpost thread iconGoogle spidering past secure server
  Robots.txt file to boot the bot?
2xbase234 12:52 am on Jan 31, 2004 (gmt 0)
newpost thread iconLogs keeping showing 404 error
  I have a robots file and it looks ok
3eesnard 12:03 am on Jan 30, 2004 (gmt 0)
newpost thread iconneed robots.txt for pages that are not linked?
  robots.txt
5rld3 5:09 pm on Jan 29, 2004 (gmt 0)
newpost thread iconSEO Robot.txt tag and robot meta tags... need advice
  Help about robot.txt and robot meta tags
3jamieabbott 2:09 pm on Jan 28, 2004 (gmt 0)
Displaying Page 61 of 71<< Back 1 ... 51 52 53 54 55 56 57 58 59 60 [ 61 ] 62 63 64 65 66 67 68 69 70 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved