homepage Welcome to WebmasterWorld Guest from 54.196.69.56
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt and template pages
  Does blocking templates prevent spiders from following links?
2butterfi 12:59 am on Feb 12, 2004 (gmt 0)
newpost thread icondisallow is allowing
  Is this done correctly?
1DavidAtWork 6:25 pm on Feb 11, 2004 (gmt 0)
newpost thread iconI don't have a robots.txt file..error 404
  Can't get good Google or Yahoo placement
6jagbitsllc 2:17 am on Feb 11, 2004 (gmt 0)
newpost thread iconWebmasterWorldForumBot?
  what is this?
2Trisha 10:33 pm on Feb 10, 2004 (gmt 0)
newpost thread iconHow to stop robots on .php files
 
3djtaverner 6:40 pm on Feb 10, 2004 (gmt 0)
newpost thread iconWhat in the world is a robot.txt?
  I need this to be stated simple for me
5Claudia_T 6:35 pm on Feb 10, 2004 (gmt 0)
newpost thread iconHelp! I need to disallow a specific query string
 
1Hissingsid 11:26 am on Feb 10, 2004 (gmt 0)
newpost thread iconPlease verify
 
2mikemcs 4:25 am on Feb 10, 2004 (gmt 0)
newpost thread iconGooglebot chasing .php tracking script
  is robots.txt enough?
2Edouard_H 12:51 am on Feb 9, 2004 (gmt 0)
newpost thread icon301 redirects & robots.txt
  What is the implication of a 301 redirect on robots.txt
2Jerry_Pannu 10:17 pm on Feb 7, 2004 (gmt 0)
newpost thread iconTesting robots.txt
  Why won't my site get spidered
2DavidAtWork 5:56 pm on Feb 5, 2004 (gmt 0)
newpost thread iconI've been looking at the WW robots.txt ...
  ...but why are these spiders disallowed?
7elgumbo 1:32 pm on Feb 4, 2004 (gmt 0)
newpost thread iconGoogleBot
  Will using robots exclusion and meta exclusion hurt me?
1mercury 9:12 pm on Feb 3, 2004 (gmt 0)
newpost thread iconSpiders that ignore or skip robots.txt
  Can bad spiders be identified?
4DavidAtWork 2:05 pm on Feb 3, 2004 (gmt 0)
newpost thread iconRobots Meta and .txt
  Where?
4admin_amoroustoys 1:34 pm on Feb 3, 2004 (gmt 0)
newpost thread iconrobots.txt used in court case
 
2transactiongeek 1:27 pm on Feb 3, 2004 (gmt 0)
newpost thread iconwhat is the "KAIST AITrc Crawler" bot?
  it ignored my "robots.txt" file and got caught in my bot trap
4stapel 12:01 pm on Feb 3, 2004 (gmt 0)
newpost thread iconIs it illegal to ignore a robots.txt?
  Could I sue google?
2transactiongeek 6:44 am on Feb 3, 2004 (gmt 0)
newpost thread iconrobots metatag
  I have noindex,follow but the page is indexed
2nevetS 4:41 am on Feb 3, 2004 (gmt 0)
newpost thread iconsave and upload robots.txt
 
4helenp 8:27 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this robots.txt blocking anybody
 
8nalavanje 2:20 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this enough
 
5dougie 11:20 am on Feb 2, 2004 (gmt 0)
newpost thread iconHow Do I Code My Robots.txt File To Disallow Subdomians?
  Is it Disallow: /apple.domain.com or Disallow: /apple/
2Christyl_Stevens 9:34 am on Feb 2, 2004 (gmt 0)
newpost thread iconasterias/2.0
  Looks for streaming media files...
6pendanticist 11:43 pm on Feb 1, 2004 (gmt 0)
newpost thread iconwill all robots crawl the sites without robot.txt?
  robot crawl, robot.txt
2johnlim 3:08 am on Feb 1, 2004 (gmt 0)
Displaying Page 61 of 71<< Back 1 ... 51 52 53 54 55 56 57 58 59 60 [ 61 ] 62 63 64 65 66 67 68 69 70 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved