homepage Welcome to WebmasterWorld Guest from 54.237.99.131
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots and redirecting
  i am wondering how a robot would act on a page with a redirect
2melanger 8:56 pm on Mar 8, 2008 (gmt 0)
newpost thread iconUnusual log entries
 
5cyberdyne 6:17 pm on Mar 8, 2008 (gmt 0)
newpost thread iconRedirect robot.txt from http to https
  how to redirect properly.
8clyde4210 2:59 pm on Mar 6, 2008 (gmt 0)
newpost thread iconUh oh, it's happening again
  robots.txt blocked out
2wesg 9:58 pm on Feb 28, 2008 (gmt 0)
newpost thread iconnew badbot: EnaBot/1.0
 
7Achernar 8:31 am on Feb 28, 2008 (gmt 0)
newpost thread iconNeed help blocking search results page
  Lots of keyword search pages indexed, some with no content
4ecmedia 2:28 pm on Feb 26, 2008 (gmt 0)
newpost thread icondisallow ALL bots.and then allow the good ones?
 
5tokey666 8:40 pm on Feb 25, 2008 (gmt 0)
newpost thread iconExcluding a stub page with robots.txt
  Dealing with trailing forward slash
4bouncybunny 12:41 am on Feb 24, 2008 (gmt 0)
newpost thread iconWhat is wrong with my Robots.txt file?
  I'd appreciate help with my Robots.txt file
7Morthan 3:07 am on Feb 22, 2008 (gmt 0)
newpost thread iconHelp with blocking URLS
 
5jaffstar 7:17 am on Feb 18, 2008 (gmt 0)
newpost thread icon301 Redirect and Search Ranking
  301 Redirect and Search Ranking
4bilalseo 6:22 am on Feb 18, 2008 (gmt 0)
newpost thread iconNo life without robots.txt file
  This may be old hat to you but it knocked me for six
18folkranger 7:17 pm on Feb 15, 2008 (gmt 0)
newpost thread iconRobots.txt and iframes
  How to disallow iframes using robots.txt?
6foxtunes 11:00 pm on Feb 14, 2008 (gmt 0)
newpost thread iconHow to Handle Cname URL
  Indexed in Google
7experienced 7:47 am on Feb 12, 2008 (gmt 0)
newpost thread iconGoogle problem with robots.txt
  Webmaster tools isn't helping.
21wesg 8:16 pm on Feb 5, 2008 (gmt 0)
newpost thread icon*/? */index.*? is a must in robots.txt
  I see /?widgetid=45 indexed
3AjiNIMC 8:17 am on Jan 30, 2008 (gmt 0)
newpost thread iconRobots.txt (/index.jsp?pg name=ccpm/lob/sublob/pgname&site=a1)
  Pattern on parameters
2Surendra 12:20 am on Jan 29, 2008 (gmt 0)
newpost thread iconDenying access to everything but the home page?
  Using robots.txt to only allow robot access to the home page
3monkeylytics 8:00 pm on Jan 28, 2008 (gmt 0)
newpost thread iconControl Robots with Meta or robots.txt
 
3driller41 12:21 pm on Jan 15, 2008 (gmt 0)
newpost thread iconRobots.txt disallowed.
 
12g1smd 12:18 am on Jan 15, 2008 (gmt 0)
newpost thread iconrobots.txt with wildcard?
 
3tranceformer 12:39 am on Jan 13, 2008 (gmt 0)
newpost thread iconDisallowing with robots.txt
  robots.txt, disallow certain urls
3jim_knopf 12:36 am on Jan 13, 2008 (gmt 0)
newpost thread iconGetting correct user agent
  getting from access log the correct user agent for robots.txt
2hessodreamy 9:03 pm on Jan 11, 2008 (gmt 0)
newpost thread iconScrapers checking robots.txt
  Spiders arrived on heels of robots.txt deployment
4timster 1:29 am on Jan 4, 2008 (gmt 0)
newpost thread icon404 Errors in Webmaster Console
  What to do with these pages?
2rowtc2 11:35 pm on Jan 3, 2008 (gmt 0)
Displaying Page 25 of 72<< Back 1 ... 15 16 17 18 19 20 21 22 23 24 [ 25 ] 26 27 28 29 30 31 32 33 34 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved