homepage Welcome to WebmasterWorld Guest from 184.73.87.85
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconOddly long robots.txt file
  Seeking explanation for weird robots.txt file
7bendede 9:31 pm on Dec 9, 2004 (gmt 0)
newpost thread iconHidden Text?
  Assistance Please....
4Kasey22 8:44 pm on Dec 7, 2004 (gmt 0)
newpost thread iconGoogle wildcard
  Is this allowed?
3maha 5:02 am on Dec 6, 2004 (gmt 0)
newpost thread iconHow do I create two robots.txt for the same directory?
  .com and .net are for two different engines but point at the same folder
3walkman 9:01 pm on Dec 2, 2004 (gmt 0)
newpost thread iconRobots.txt Newbie Question
 
4paulpsd 12:01 am on Dec 2, 2004 (gmt 0)
newpost thread iconredirecting domain
 
3tomasz 9:31 pm on Dec 1, 2004 (gmt 0)
newpost thread iconsubdomains and robots.txt
 
2gettin_there 8:54 pm on Dec 1, 2004 (gmt 0)
newpost thread iconUser-agent: ...
 
3gettin_there 3:06 pm on Dec 1, 2004 (gmt 0)
newpost thread iconCrawling of non-linked pages
  Over 18's area
3theleveller 2:43 pm on Dec 1, 2004 (gmt 0)
newpost thread iconAsk Jeeves Question
  Need to ban a bot
2Marcus_Aurelius 10:13 am on Dec 1, 2004 (gmt 0)
newpost thread iconWebmaster World's Robots.txt
  Why don't they allow msnbot?
6Rollo 6:11 pm on Nov 30, 2004 (gmt 0)
newpost thread iconContent of Robots.txt files
  You wouldn't believe what some hosts think should be in a Robots.txt file!
9AlucardSpyderWriter 8:17 pm on Nov 28, 2004 (gmt 0)
newpost thread iconHow would I disallow Google...
  ...and allow all others?
3Rollo 5:35 pm on Nov 28, 2004 (gmt 0)
newpost thread iconRobot.txt specialists: will Disallow: /? work?
  need to block google from ...com/?tracking.
1walkman 7:46 am on Nov 28, 2004 (gmt 0)
newpost thread iconrobots.txt-newbie needs help
  Is this correct?
4cybervette 5:18 pm on Nov 26, 2004 (gmt 0)
newpost thread iconChanging your Mind with robots.txt
  Will googlebot come back to the party?
2Spine 4:41 pm on Nov 26, 2004 (gmt 0)
newpost thread icondynamic folders
 
1apan 12:24 pm on Nov 25, 2004 (gmt 0)
newpost thread iconrobots.txt validator problem?
  it is reading my index.htm file?
2mbatta 11:34 pm on Nov 22, 2004 (gmt 0)
newpost thread iconSyntax: User-agent: Mediapartners-Google*
 
5jhubb 3:21 pm on Nov 22, 2004 (gmt 0)
newpost thread iconrobots.txt question
  I asked whether does this, but the engine spider ip address has any to use
2ericchui 11:18 am on Nov 22, 2004 (gmt 0)
newpost thread iconProtecting robots.txt file from bad guys
  Is it enough to passwoed protect those dirs?
3damlag 4:10 pm on Nov 21, 2004 (gmt 0)
newpost thread iconHow to Exclude from Yahoo/MSN results!
  I want to exclude underconstruction site
2fm101 1:49 am on Nov 21, 2004 (gmt 0)
newpost thread iconHow to block all subdirectories?
 
2TerryMc 7:05 pm on Nov 17, 2004 (gmt 0)
newpost thread iconHow to exclude dynamic URLS?
  So Search engines dont get bogged down.
7Tech2004 4:54 am on Nov 16, 2004 (gmt 0)
newpost thread iconEffect of Robots.txt
 
5seolancer 5:31 am on Nov 15, 2004 (gmt 0)
Displaying Page 52 of 71<< Back 1 ... 42 43 44 45 46 47 48 49 50 51 [ 52 ] 53 54 55 56 57 58 59 60 61 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved