homepage Welcome to WebmasterWorld Guest from 54.227.34.0
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconexclude subdirectories
 
3seoerer 11:08 pm on Jan 7, 2005 (gmt 0)
newpost thread iconRobot.txt Generator
  should i put the results on my website?
3rj87uk 10:54 am on Jan 6, 2005 (gmt 0)
newpost thread iconNot sure how to handle space character
 
5Jon_King 1:48 am on Jan 6, 2005 (gmt 0)
newpost thread iconProblem of syntax and ordering
 
1esllou 11:50 am on Jan 4, 2005 (gmt 0)
newpost thread iconWill robots.txt affect IE or other browsers?
  A little confuse...
2iProgram 4:34 am on Jan 4, 2005 (gmt 0)
newpost thread iconwhat are the names of the image bots?
  I can't ban them if I don't know their names
8stapel 5:17 pm on Jan 2, 2005 (gmt 0)
newpost thread iconBlocking images folder from spiders
  Good or bad idea?
2twist 10:17 pm on Jan 1, 2005 (gmt 0)
newpost thread iconSudden Increase in robots.txt Hits
 
3Rob_Moore 10:04 pm on Dec 26, 2004 (gmt 0)
newpost thread iconKeeping spiders off co-branded sites
  how to keep spiders from indexing our partner / co-branded sites
4BillingOrchard 9:41 pm on Dec 23, 2004 (gmt 0)
newpost thread iconAbout robots.txt file
  Confusion in robots.txt file...i need hep..
7venuc 5:32 am on Dec 21, 2004 (gmt 0)
newpost thread iconGooglebot Spoofing
  How do you tell the real googlebot from spoofers?
3teaperson 6:29 pm on Dec 20, 2004 (gmt 0)
newpost thread iconTricky Robots Txt Help,
  Two sites one robot
3badboybart 2:50 pm on Dec 20, 2004 (gmt 0)
newpost thread iconredirect without confusing spiders
 
1houseofsecrets 4:55 pm on Dec 13, 2004 (gmt 0)
newpost thread iconRobot.txt file deleted by accident what now?
 
2Hoppo 11:15 pm on Dec 12, 2004 (gmt 0)
newpost thread iconrobots.txt and meta tags
  can I use both?
2kdawg 11:23 pm on Dec 11, 2004 (gmt 0)
newpost thread iconOddly long robots.txt file
  Seeking explanation for weird robots.txt file
7bendede 9:31 pm on Dec 9, 2004 (gmt 0)
newpost thread iconHidden Text?
  Assistance Please....
4Kasey22 8:44 pm on Dec 7, 2004 (gmt 0)
newpost thread iconGoogle wildcard
  Is this allowed?
3maha 5:02 am on Dec 6, 2004 (gmt 0)
newpost thread iconHow do I create two robots.txt for the same directory?
  .com and .net are for two different engines but point at the same folder
3walkman 9:01 pm on Dec 2, 2004 (gmt 0)
newpost thread iconRobots.txt Newbie Question
 
4paulpsd 12:01 am on Dec 2, 2004 (gmt 0)
newpost thread iconredirecting domain
 
3tomasz 9:31 pm on Dec 1, 2004 (gmt 0)
newpost thread iconsubdomains and robots.txt
 
2gettin_there 8:54 pm on Dec 1, 2004 (gmt 0)
newpost thread iconUser-agent: ...
 
3gettin_there 3:06 pm on Dec 1, 2004 (gmt 0)
newpost thread iconCrawling of non-linked pages
  Over 18's area
3theleveller 2:43 pm on Dec 1, 2004 (gmt 0)
newpost thread iconAsk Jeeves Question
  Need to ban a bot
2Marcus_Aurelius 10:13 am on Dec 1, 2004 (gmt 0)
Displaying Page 52 of 72<< Back 1 ... 42 43 44 45 46 47 48 49 50 51 [ 52 ] 53 54 55 56 57 58 59 60 61 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved