homepage Welcome to WebmasterWorld Guest from 54.205.254.108
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconSudden Increase in robots.txt Hits
 
3Rob_Moore 10:04 pm on Dec 26, 2004 (gmt 0)
newpost thread iconKeeping spiders off co-branded sites
  how to keep spiders from indexing our partner / co-branded sites
4BillingOrchard 9:41 pm on Dec 23, 2004 (gmt 0)
newpost thread iconAbout robots.txt file
  Confusion in robots.txt file...i need hep..
7venuc 5:32 am on Dec 21, 2004 (gmt 0)
newpost thread iconGooglebot Spoofing
  How do you tell the real googlebot from spoofers?
3teaperson 6:29 pm on Dec 20, 2004 (gmt 0)
newpost thread iconTricky Robots Txt Help,
  Two sites one robot
3badboybart 2:50 pm on Dec 20, 2004 (gmt 0)
newpost thread iconredirect without confusing spiders
 
1houseofsecrets 4:55 pm on Dec 13, 2004 (gmt 0)
newpost thread iconRobot.txt file deleted by accident what now?
 
2Hoppo 11:15 pm on Dec 12, 2004 (gmt 0)
newpost thread iconrobots.txt and meta tags
  can I use both?
2kdawg 11:23 pm on Dec 11, 2004 (gmt 0)
newpost thread iconOddly long robots.txt file
  Seeking explanation for weird robots.txt file
7bendede 9:31 pm on Dec 9, 2004 (gmt 0)
newpost thread iconHidden Text?
  Assistance Please....
4Kasey22 8:44 pm on Dec 7, 2004 (gmt 0)
newpost thread iconGoogle wildcard
  Is this allowed?
3maha 5:02 am on Dec 6, 2004 (gmt 0)
newpost thread iconHow do I create two robots.txt for the same directory?
  .com and .net are for two different engines but point at the same folder
3walkman 9:01 pm on Dec 2, 2004 (gmt 0)
newpost thread iconRobots.txt Newbie Question
 
4paulpsd 12:01 am on Dec 2, 2004 (gmt 0)
newpost thread iconredirecting domain
 
3tomasz 9:31 pm on Dec 1, 2004 (gmt 0)
newpost thread iconsubdomains and robots.txt
 
2gettin_there 8:54 pm on Dec 1, 2004 (gmt 0)
newpost thread iconUser-agent: ...
 
3gettin_there 3:06 pm on Dec 1, 2004 (gmt 0)
newpost thread iconCrawling of non-linked pages
  Over 18's area
3theleveller 2:43 pm on Dec 1, 2004 (gmt 0)
newpost thread iconAsk Jeeves Question
  Need to ban a bot
2Marcus_Aurelius 10:13 am on Dec 1, 2004 (gmt 0)
newpost thread iconWebmaster World's Robots.txt
  Why don't they allow msnbot?
6Rollo 6:11 pm on Nov 30, 2004 (gmt 0)
newpost thread iconContent of Robots.txt files
  You wouldn't believe what some hosts think should be in a Robots.txt file!
9AlucardSpyderWriter 8:17 pm on Nov 28, 2004 (gmt 0)
newpost thread iconHow would I disallow Google...
  ...and allow all others?
3Rollo 5:35 pm on Nov 28, 2004 (gmt 0)
newpost thread iconRobot.txt specialists: will Disallow: /? work?
  need to block google from ...com/?tracking.
1walkman 7:46 am on Nov 28, 2004 (gmt 0)
newpost thread iconrobots.txt-newbie needs help
  Is this correct?
4cybervette 5:18 pm on Nov 26, 2004 (gmt 0)
newpost thread iconChanging your Mind with robots.txt
  Will googlebot come back to the party?
2Spine 4:41 pm on Nov 26, 2004 (gmt 0)
newpost thread icondynamic folders
 
1apan 12:24 pm on Nov 25, 2004 (gmt 0)
Displaying Page 52 of 71<< Back 1 ... 42 43 44 45 46 47 48 49 50 51 [ 52 ] 53 54 55 56 57 58 59 60 61 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved