homepage Welcome to WebmasterWorld Guest from 54.227.5.234
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconIs this valid? wildcard in robots
  Disallow: /*/edit.html
3rudyten 1:47 pm on Apr 20, 2008 (gmt 0)
newpost thread iconWill spiders follow links on URL's blocked by robots.txt
 
3deanw 6:34 pm on Apr 16, 2008 (gmt 0)
newpost thread iconSecurity Challenges
 
4pageoneresults 3:49 pm on Apr 2, 2008 (gmt 0)
newpost thread iconDumbest question eva: Do all major SEs support the Google guidelines?
  I am testing mine on G's WebmasterCentral
2walkman 6:42 pm on Mar 26, 2008 (gmt 0)
newpost thread iconIs there any way to block everyone except.
  for Google, Yahoo etc?
2mmiller 9:41 am on Mar 21, 2008 (gmt 0)
newpost thread iconneed to block. and don't know - pls help
  need help with blocking - Malware Ref
4Reilly 7:55 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow to block original + localized pages?
  robots.txt localized
1vanguard2000 6:19 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow long before bots re-visit after they were told they are not allowed?
 
3boromir27 3:32 am on Mar 17, 2008 (gmt 0)
newpost thread iconInktomi User-Agent blocked
  Inktomi blocked
6cyberdyne 8:36 am on Mar 14, 2008 (gmt 0)
newpost thread iconrobots.txt - Disallow/Allow
 
7eyalkattan 1:14 am on Mar 13, 2008 (gmt 0)
newpost thread iconrobots and redirecting
  i am wondering how a robot would act on a page with a redirect
2melanger 8:56 pm on Mar 8, 2008 (gmt 0)
newpost thread iconUnusual log entries
 
5cyberdyne 6:17 pm on Mar 8, 2008 (gmt 0)
newpost thread iconRedirect robot.txt from http to https
  how to redirect properly.
8clyde4210 2:59 pm on Mar 6, 2008 (gmt 0)
newpost thread iconUh oh, it's happening again
  robots.txt blocked out
2wesg 9:58 pm on Feb 28, 2008 (gmt 0)
newpost thread iconnew badbot: EnaBot/1.0
 
7Achernar 8:31 am on Feb 28, 2008 (gmt 0)
newpost thread iconNeed help blocking search results page
  Lots of keyword search pages indexed, some with no content
4ecmedia 2:28 pm on Feb 26, 2008 (gmt 0)
newpost thread icondisallow ALL bots.and then allow the good ones?
 
5tokey666 8:40 pm on Feb 25, 2008 (gmt 0)
newpost thread iconExcluding a stub page with robots.txt
  Dealing with trailing forward slash
4bouncybunny 12:41 am on Feb 24, 2008 (gmt 0)
newpost thread iconWhat is wrong with my Robots.txt file?
  I'd appreciate help with my Robots.txt file
7Morthan 3:07 am on Feb 22, 2008 (gmt 0)
newpost thread iconHelp with blocking URLS
 
5jaffstar 7:17 am on Feb 18, 2008 (gmt 0)
newpost thread icon301 Redirect and Search Ranking
  301 Redirect and Search Ranking
4bilalseo 6:22 am on Feb 18, 2008 (gmt 0)
newpost thread iconNo life without robots.txt file
  This may be old hat to you but it knocked me for six
18folkranger 7:17 pm on Feb 15, 2008 (gmt 0)
newpost thread iconRobots.txt and iframes
  How to disallow iframes using robots.txt?
6foxtunes 11:00 pm on Feb 14, 2008 (gmt 0)
newpost thread iconHow to Handle Cname URL
  Indexed in Google
7experienced 7:47 am on Feb 12, 2008 (gmt 0)
newpost thread iconGoogle problem with robots.txt
  Webmaster tools isn't helping.
21wesg 8:16 pm on Feb 5, 2008 (gmt 0)
Displaying Page 24 of 71<< Back 1 ... 14 15 16 17 18 19 20 21 22 23 [ 24 ] 25 26 27 28 29 30 31 32 33 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved