homepage Welcome to WebmasterWorld Guest from 54.234.2.94
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDo I need to
 
2ronhollin 6:46 am on Feb 16, 2004 (gmt 0)
newpost thread iconHow important is having a properly configured robots?
 
4budbiss 3:45 pm on Feb 14, 2004 (gmt 0)
newpost thread iconDOS Line Enders
  How to download sample robots.txt files
3webman2 10:30 am on Feb 14, 2004 (gmt 0)
newpost thread iconrobots.txt and template pages
  Does blocking templates prevent spiders from following links?
2butterfi 12:59 am on Feb 12, 2004 (gmt 0)
newpost thread icondisallow is allowing
  Is this done correctly?
1DavidAtWork 6:25 pm on Feb 11, 2004 (gmt 0)
newpost thread iconI don't have a robots.txt file..error 404
  Can't get good Google or Yahoo placement
6jagbitsllc 2:17 am on Feb 11, 2004 (gmt 0)
newpost thread iconWebmasterWorldForumBot?
  what is this?
2Trisha 10:33 pm on Feb 10, 2004 (gmt 0)
newpost thread iconHow to stop robots on .php files
 
3djtaverner 6:40 pm on Feb 10, 2004 (gmt 0)
newpost thread iconWhat in the world is a robot.txt?
  I need this to be stated simple for me
5Claudia_T 6:35 pm on Feb 10, 2004 (gmt 0)
newpost thread iconHelp! I need to disallow a specific query string
 
1Hissingsid 11:26 am on Feb 10, 2004 (gmt 0)
newpost thread iconPlease verify
 
2mikemcs 4:25 am on Feb 10, 2004 (gmt 0)
newpost thread iconGooglebot chasing .php tracking script
  is robots.txt enough?
2Edouard_H 12:51 am on Feb 9, 2004 (gmt 0)
newpost thread icon301 redirects & robots.txt
  What is the implication of a 301 redirect on robots.txt
2Jerry_Pannu 10:17 pm on Feb 7, 2004 (gmt 0)
newpost thread iconTesting robots.txt
  Why won't my site get spidered
2DavidAtWork 5:56 pm on Feb 5, 2004 (gmt 0)
newpost thread iconI've been looking at the WW robots.txt ...
  ...but why are these spiders disallowed?
7elgumbo 1:32 pm on Feb 4, 2004 (gmt 0)
newpost thread iconGoogleBot
  Will using robots exclusion and meta exclusion hurt me?
1mercury 9:12 pm on Feb 3, 2004 (gmt 0)
newpost thread iconSpiders that ignore or skip robots.txt
  Can bad spiders be identified?
4DavidAtWork 2:05 pm on Feb 3, 2004 (gmt 0)
newpost thread iconRobots Meta and .txt
  Where?
4admin_amoroustoys 1:34 pm on Feb 3, 2004 (gmt 0)
newpost thread iconrobots.txt used in court case
 
2transactiongeek 1:27 pm on Feb 3, 2004 (gmt 0)
newpost thread iconwhat is the "KAIST AITrc Crawler" bot?
  it ignored my "robots.txt" file and got caught in my bot trap
4stapel 12:01 pm on Feb 3, 2004 (gmt 0)
newpost thread iconIs it illegal to ignore a robots.txt?
  Could I sue google?
2transactiongeek 6:44 am on Feb 3, 2004 (gmt 0)
newpost thread iconrobots metatag
  I have noindex,follow but the page is indexed
2nevetS 4:41 am on Feb 3, 2004 (gmt 0)
newpost thread iconsave and upload robots.txt
 
4helenp 8:27 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this robots.txt blocking anybody
 
8nalavanje 2:20 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this enough
 
5dougie 11:20 am on Feb 2, 2004 (gmt 0)
Displaying Page 61 of 71<< Back 1 ... 51 52 53 54 55 56 57 58 59 60 [ 61 ] 62 63 64 65 66 67 68 69 70 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved