homepage Welcome to WebmasterWorld Guest from 54.166.45.31
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconredirecting domain
 
3tomasz 9:31 pm on Dec 1, 2004 (gmt 0)
newpost thread iconsubdomains and robots.txt
 
2gettin_there 8:54 pm on Dec 1, 2004 (gmt 0)
newpost thread iconUser-agent: ...
 
3gettin_there 3:06 pm on Dec 1, 2004 (gmt 0)
newpost thread iconCrawling of non-linked pages
  Over 18's area
3theleveller 2:43 pm on Dec 1, 2004 (gmt 0)
newpost thread iconAsk Jeeves Question
  Need to ban a bot
2Marcus_Aurelius 10:13 am on Dec 1, 2004 (gmt 0)
newpost thread iconWebmaster World's Robots.txt
  Why don't they allow msnbot?
6Rollo 6:11 pm on Nov 30, 2004 (gmt 0)
newpost thread iconContent of Robots.txt files
  You wouldn't believe what some hosts think should be in a Robots.txt file!
9AlucardSpyderWriter 8:17 pm on Nov 28, 2004 (gmt 0)
newpost thread iconHow would I disallow Google...
  ...and allow all others?
3Rollo 5:35 pm on Nov 28, 2004 (gmt 0)
newpost thread iconRobot.txt specialists: will Disallow: /? work?
  need to block google from ...com/?tracking.
1walkman 7:46 am on Nov 28, 2004 (gmt 0)
newpost thread iconrobots.txt-newbie needs help
  Is this correct?
4cybervette 5:18 pm on Nov 26, 2004 (gmt 0)
newpost thread iconChanging your Mind with robots.txt
  Will googlebot come back to the party?
2Spine 4:41 pm on Nov 26, 2004 (gmt 0)
newpost thread icondynamic folders
 
1apan 12:24 pm on Nov 25, 2004 (gmt 0)
newpost thread iconrobots.txt validator problem?
  it is reading my index.htm file?
2mbatta 11:34 pm on Nov 22, 2004 (gmt 0)
newpost thread iconSyntax: User-agent: Mediapartners-Google*
 
5jhubb 3:21 pm on Nov 22, 2004 (gmt 0)
newpost thread iconrobots.txt question
  I asked whether does this, but the engine spider ip address has any to use
2ericchui 11:18 am on Nov 22, 2004 (gmt 0)
newpost thread iconProtecting robots.txt file from bad guys
  Is it enough to passwoed protect those dirs?
3damlag 4:10 pm on Nov 21, 2004 (gmt 0)
newpost thread iconHow to Exclude from Yahoo/MSN results!
  I want to exclude underconstruction site
2fm101 1:49 am on Nov 21, 2004 (gmt 0)
newpost thread iconHow to block all subdirectories?
 
2TerryMc 7:05 pm on Nov 17, 2004 (gmt 0)
newpost thread iconHow to exclude dynamic URLS?
  So Search engines dont get bogged down.
7Tech2004 4:54 am on Nov 16, 2004 (gmt 0)
newpost thread iconEffect of Robots.txt
 
5seolancer 5:31 am on Nov 15, 2004 (gmt 0)
newpost thread iconrobots.txt - newbie needs help
 
5geezer_uk 5:39 am on Nov 10, 2004 (gmt 0)
newpost thread iconexcluding cgi-bin
  is this right
2rbacal 2:59 am on Nov 10, 2004 (gmt 0)
newpost thread iconIs robot.txt a must for good SEO?
  --whats the purpose of robot.txt
4redleaf 12:24 am on Nov 9, 2004 (gmt 0)
newpost thread iconWeird 404 error hitting on robot.txt
  anyone experienced this before?
3redleaf 9:30 am on Nov 8, 2004 (gmt 0)
newpost thread iconrobots.txt and free web hosting
 
3rhodopsin 7:46 am on Nov 8, 2004 (gmt 0)
Displaying Page 53 of 72<< Back 1 ... 43 44 45 46 47 48 49 50 51 52 [ 53 ] 54 55 56 57 58 59 60 61 62 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved