homepage Welcome to WebmasterWorld Guest from 54.226.10.234
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconWebmaster World's Robots.txt
  Why don't they allow msnbot?
6Rollo 6:11 pm on Nov 30, 2004 (gmt 0)
newpost thread iconContent of Robots.txt files
  You wouldn't believe what some hosts think should be in a Robots.txt file!
9AlucardSpyderWriter 8:17 pm on Nov 28, 2004 (gmt 0)
newpost thread iconHow would I disallow Google...
  ...and allow all others?
3Rollo 5:35 pm on Nov 28, 2004 (gmt 0)
newpost thread iconRobot.txt specialists: will Disallow: /? work?
  need to block google from ...com/?tracking.
1walkman 7:46 am on Nov 28, 2004 (gmt 0)
newpost thread iconrobots.txt-newbie needs help
  Is this correct?
4cybervette 5:18 pm on Nov 26, 2004 (gmt 0)
newpost thread iconChanging your Mind with robots.txt
  Will googlebot come back to the party?
2Spine 4:41 pm on Nov 26, 2004 (gmt 0)
newpost thread icondynamic folders
 
1apan 12:24 pm on Nov 25, 2004 (gmt 0)
newpost thread iconrobots.txt validator problem?
  it is reading my index.htm file?
2mbatta 11:34 pm on Nov 22, 2004 (gmt 0)
newpost thread iconSyntax: User-agent: Mediapartners-Google*
 
5jhubb 3:21 pm on Nov 22, 2004 (gmt 0)
newpost thread iconrobots.txt question
  I asked whether does this, but the engine spider ip address has any to use
2ericchui 11:18 am on Nov 22, 2004 (gmt 0)
newpost thread iconProtecting robots.txt file from bad guys
  Is it enough to passwoed protect those dirs?
3damlag 4:10 pm on Nov 21, 2004 (gmt 0)
newpost thread iconHow to Exclude from Yahoo/MSN results!
  I want to exclude underconstruction site
2fm101 1:49 am on Nov 21, 2004 (gmt 0)
newpost thread iconHow to block all subdirectories?
 
2TerryMc 7:05 pm on Nov 17, 2004 (gmt 0)
newpost thread iconHow to exclude dynamic URLS?
  So Search engines dont get bogged down.
7Tech2004 4:54 am on Nov 16, 2004 (gmt 0)
newpost thread iconEffect of Robots.txt
 
5seolancer 5:31 am on Nov 15, 2004 (gmt 0)
newpost thread iconrobots.txt - newbie needs help
 
5geezer_uk 5:39 am on Nov 10, 2004 (gmt 0)
newpost thread iconexcluding cgi-bin
  is this right
2rbacal 2:59 am on Nov 10, 2004 (gmt 0)
newpost thread iconIs robot.txt a must for good SEO?
  --whats the purpose of robot.txt
4redleaf 12:24 am on Nov 9, 2004 (gmt 0)
newpost thread iconWeird 404 error hitting on robot.txt
  anyone experienced this before?
3redleaf 9:30 am on Nov 8, 2004 (gmt 0)
newpost thread iconrobots.txt and free web hosting
 
3rhodopsin 7:46 am on Nov 8, 2004 (gmt 0)
newpost thread iconPossible to have a page disalowed by robots.txt but indexed by adsense
  Possible to do this?
2brakkar 9:16 pm on Nov 5, 2004 (gmt 0)
newpost thread iconProtocol interpretation
  Peerbot or I misunderstood the robot exclusion protocol
3WebJoe 9:11 pm on Nov 5, 2004 (gmt 0)
newpost thread iconNeed help about File Robots.txt
  how do I know spiders visit to my site or not?
3viva 10:15 am on Nov 4, 2004 (gmt 0)
newpost thread iconwhat happen with robot exclusion and iframes?
  could it be a problem to exclude content that appears inside a home iframe?
2mgorricho 5:03 pm on Nov 3, 2004 (gmt 0)
newpost thread iconprotocol extension resource control
  proposed extension to robots.txt protocol allowing resource control
1copsewood 11:29 am on Nov 2, 2004 (gmt 0)
Displaying Page 53 of 72<< Back 1 ... 43 44 45 46 47 48 49 50 51 52 [ 53 ] 54 55 56 57 58 59 60 61 62 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved