homepage Welcome to WebmasterWorld Guest from 54.197.111.87
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconNeed help blocking search results page
  Lots of keyword search pages indexed, some with no content
4ecmedia 2:28 pm on Feb 26, 2008 (gmt 0)
newpost thread icondisallow ALL bots.and then allow the good ones?
 
5tokey666 8:40 pm on Feb 25, 2008 (gmt 0)
newpost thread iconExcluding a stub page with robots.txt
  Dealing with trailing forward slash
4bouncybunny 12:41 am on Feb 24, 2008 (gmt 0)
newpost thread iconWhat is wrong with my Robots.txt file?
  I'd appreciate help with my Robots.txt file
7Morthan 3:07 am on Feb 22, 2008 (gmt 0)
newpost thread iconHelp with blocking URLS
 
5jaffstar 7:17 am on Feb 18, 2008 (gmt 0)
newpost thread icon301 Redirect and Search Ranking
  301 Redirect and Search Ranking
4bilalseo 6:22 am on Feb 18, 2008 (gmt 0)
newpost thread iconNo life without robots.txt file
  This may be old hat to you but it knocked me for six
18folkranger 7:17 pm on Feb 15, 2008 (gmt 0)
newpost thread iconRobots.txt and iframes
  How to disallow iframes using robots.txt?
6foxtunes 11:00 pm on Feb 14, 2008 (gmt 0)
newpost thread iconHow to Handle Cname URL
  Indexed in Google
7experienced 7:47 am on Feb 12, 2008 (gmt 0)
newpost thread iconGoogle problem with robots.txt
  Webmaster tools isn't helping.
21wesg 8:16 pm on Feb 5, 2008 (gmt 0)
newpost thread icon*/? */index.*? is a must in robots.txt
  I see /?widgetid=45 indexed
3AjiNIMC 8:17 am on Jan 30, 2008 (gmt 0)
newpost thread iconRobots.txt (/index.jsp?pg name=ccpm/lob/sublob/pgname&site=a1)
  Pattern on parameters
2Surendra 12:20 am on Jan 29, 2008 (gmt 0)
newpost thread iconDenying access to everything but the home page?
  Using robots.txt to only allow robot access to the home page
3monkeylytics 8:00 pm on Jan 28, 2008 (gmt 0)
newpost thread iconControl Robots with Meta or robots.txt
 
3driller41 12:21 pm on Jan 15, 2008 (gmt 0)
newpost thread iconRobots.txt disallowed.
 
12g1smd 12:18 am on Jan 15, 2008 (gmt 0)
newpost thread iconrobots.txt with wildcard?
 
3tranceformer 12:39 am on Jan 13, 2008 (gmt 0)
newpost thread iconDisallowing with robots.txt
  robots.txt, disallow certain urls
3jim_knopf 12:36 am on Jan 13, 2008 (gmt 0)
newpost thread iconGetting correct user agent
  getting from access log the correct user agent for robots.txt
2hessodreamy 9:03 pm on Jan 11, 2008 (gmt 0)
newpost thread iconScrapers checking robots.txt
  Spiders arrived on heels of robots.txt deployment
4timster 1:29 am on Jan 4, 2008 (gmt 0)
newpost thread icon404 Errors in Webmaster Console
  What to do with these pages?
2rowtc2 11:35 pm on Jan 3, 2008 (gmt 0)
newpost thread icondirectory and subdirectory access
  allowing access to subdirectory but not main directory
3Mr_Cool_Dude 6:11 pm on Jan 2, 2008 (gmt 0)
newpost thread iconrobots.txt question
  robots.txt examples
2jim_knopf 5:43 pm on Jan 2, 2008 (gmt 0)
newpost thread iconReference robots.txt file?
  Does anyone have one?
3mmiller 11:30 pm on Dec 25, 2007 (gmt 0)
newpost thread iconhow to manage cname in robot
  don't was to be indexed
4experienced 2:10 pm on Dec 21, 2007 (gmt 0)
newpost thread icondisallow vs ban in .htaccess
 
12joergnw10 11:10 am on Dec 16, 2007 (gmt 0)
Displaying Page 25 of 72<< Back 1 ... 15 16 17 18 19 20 21 22 23 24 [ 25 ] 26 27 28 29 30 31 32 33 34 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved