homepage Welcome to WebmasterWorld Guest from 54.198.130.203
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconredirect without confusing spiders
 
1houseofsecrets 4:55 pm on Dec 13, 2004 (gmt 0)
newpost thread iconRobot.txt file deleted by accident what now?
 
2Hoppo 11:15 pm on Dec 12, 2004 (gmt 0)
newpost thread iconrobots.txt and meta tags
  can I use both?
2kdawg 11:23 pm on Dec 11, 2004 (gmt 0)
newpost thread iconOddly long robots.txt file
  Seeking explanation for weird robots.txt file
7bendede 9:31 pm on Dec 9, 2004 (gmt 0)
newpost thread iconHidden Text?
  Assistance Please....
4Kasey22 8:44 pm on Dec 7, 2004 (gmt 0)
newpost thread iconGoogle wildcard
  Is this allowed?
3maha 5:02 am on Dec 6, 2004 (gmt 0)
newpost thread iconHow do I create two robots.txt for the same directory?
  .com and .net are for two different engines but point at the same folder
3walkman 9:01 pm on Dec 2, 2004 (gmt 0)
newpost thread iconRobots.txt Newbie Question
 
4paulpsd 12:01 am on Dec 2, 2004 (gmt 0)
newpost thread iconredirecting domain
 
3tomasz 9:31 pm on Dec 1, 2004 (gmt 0)
newpost thread iconsubdomains and robots.txt
 
2gettin_there 8:54 pm on Dec 1, 2004 (gmt 0)
newpost thread iconUser-agent: ...
 
3gettin_there 3:06 pm on Dec 1, 2004 (gmt 0)
newpost thread iconCrawling of non-linked pages
  Over 18's area
3theleveller 2:43 pm on Dec 1, 2004 (gmt 0)
newpost thread iconAsk Jeeves Question
  Need to ban a bot
2Marcus_Aurelius 10:13 am on Dec 1, 2004 (gmt 0)
newpost thread iconWebmaster World's Robots.txt
  Why don't they allow msnbot?
6Rollo 6:11 pm on Nov 30, 2004 (gmt 0)
newpost thread iconContent of Robots.txt files
  You wouldn't believe what some hosts think should be in a Robots.txt file!
9AlucardSpyderWriter 8:17 pm on Nov 28, 2004 (gmt 0)
newpost thread iconHow would I disallow Google...
  ...and allow all others?
3Rollo 5:35 pm on Nov 28, 2004 (gmt 0)
newpost thread iconRobot.txt specialists: will Disallow: /? work?
  need to block google from ...com/?tracking.
1walkman 7:46 am on Nov 28, 2004 (gmt 0)
newpost thread iconrobots.txt-newbie needs help
  Is this correct?
4cybervette 5:18 pm on Nov 26, 2004 (gmt 0)
newpost thread iconChanging your Mind with robots.txt
  Will googlebot come back to the party?
2Spine 4:41 pm on Nov 26, 2004 (gmt 0)
newpost thread icondynamic folders
 
1apan 12:24 pm on Nov 25, 2004 (gmt 0)
newpost thread iconrobots.txt validator problem?
  it is reading my index.htm file?
2mbatta 11:34 pm on Nov 22, 2004 (gmt 0)
newpost thread iconSyntax: User-agent: Mediapartners-Google*
 
5jhubb 3:21 pm on Nov 22, 2004 (gmt 0)
newpost thread iconrobots.txt question
  I asked whether does this, but the engine spider ip address has any to use
2ericchui 11:18 am on Nov 22, 2004 (gmt 0)
newpost thread iconProtecting robots.txt file from bad guys
  Is it enough to passwoed protect those dirs?
3damlag 4:10 pm on Nov 21, 2004 (gmt 0)
newpost thread iconHow to Exclude from Yahoo/MSN results!
  I want to exclude underconstruction site
2fm101 1:49 am on Nov 21, 2004 (gmt 0)
Displaying Page 52 of 71<< Back 1 ... 42 43 44 45 46 47 48 49 50 51 [ 52 ] 53 54 55 56 57 58 59 60 61 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved