homepage Welcome to WebmasterWorld Guest from 54.226.93.128
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt syntax
  '$' and '?'
2sssweb 8:40 pm on Nov 11, 2006 (gmt 0)
newpost thread iconrobots.txt help
 
2miracle 7:16 pm on Nov 8, 2006 (gmt 0)
newpost thread iconLink Checker / Site map generator
  One that observes robots.txt and meta tags
5fish_eye 5:33 am on Nov 7, 2006 (gmt 0)
newpost thread iconDisallow by ISP
  Where to get a list of ISPs and the countries they are in?
1youfoundjake 7:13 pm on Nov 3, 2006 (gmt 0)
newpost thread iconMade the mistake of putting sub folders I want allowed
  disallow everthing but?
2Sunnyvale 7:07 pm on Oct 31, 2006 (gmt 0)
newpost thread iconGoogle remove url tool for supplemental urls a waste of time?
  Should I bother removing duplicate supplementals?
3Tonerman 4:20 pm on Oct 30, 2006 (gmt 0)
newpost thread iconallow major engines, block the fleas
  and to add further control
5youfoundjake 5:16 pm on Oct 26, 2006 (gmt 0)
newpost thread iconDissallowing Dynamic Content
  Blocking "index.php?"
2trillianjedi 6:58 pm on Oct 25, 2006 (gmt 0)
newpost thread iconCombining htaccess and robots.txt?
 
7Lady18 11:37 pm on Oct 24, 2006 (gmt 0)
newpost thread icondisallowing a 'staging server'
 
2rden17 4:11 pm on Oct 24, 2006 (gmt 0)
newpost thread icon.htaccess
 
2ssgumby 9:53 pm on Oct 21, 2006 (gmt 0)
newpost thread iconcloaking robots.txt
  to avoid hacking
4idolw 12:26 pm on Oct 18, 2006 (gmt 0)
newpost thread iconHow to deal with a robots.txt hack attack
  Guidelines to mitigate the damage from an attack
3Whitey 10:17 pm on Oct 17, 2006 (gmt 0)
newpost thread iconMainseek bot
  this bot is ripping through our site
2ssgumby 6:36 pm on Oct 14, 2006 (gmt 0)
newpost thread iconGoogle indexing weird links
  ones with // in them
4netchicken1 12:54 am on Oct 13, 2006 (gmt 0)
newpost thread iconPlease welcome your new moderator: goodroi
 
20engine 6:35 am on Oct 11, 2006 (gmt 0)
newpost thread iconrobots.txt problem and subdirs / wildcards
 
2disgust 1:44 pm on Oct 10, 2006 (gmt 0)
newpost thread iconGoogle Sitemaps Robots.txt Question
 
2Tonerman 5:31 pm on Oct 9, 2006 (gmt 0)
newpost thread iconNaming da' bots
  Same place, different face
1Reno 11:58 am on Oct 7, 2006 (gmt 0)
newpost thread iconTravelLazerBot
  Anyone seen this bot before?
2jexx 7:44 pm on Oct 3, 2006 (gmt 0)
newpost thread iconDisallow Coldfusion Pages Only
  Stop cfm pages from being crawled
2stcrim 3:18 pm on Oct 3, 2006 (gmt 0)
newpost thread iconRemoving specific file type
 
8MisterT 11:40 pm on Sep 28, 2006 (gmt 0)
newpost thread iconRobots - Allowing them
  rather than disallow
2SanDiegoPaul 10:30 pm on Sep 28, 2006 (gmt 0)
newpost thread iconRobots.txt & meta
  Robots meta tage or just a robots.txt
4pazang 7:45 pm on Sep 27, 2006 (gmt 0)
newpost thread iconDo excluded pages still count as links-in
 
3johnlod 8:14 pm on Sep 26, 2006 (gmt 0)
Displaying Page 33 of 71<< Back 1 ... 23 24 25 26 27 28 29 30 31 32 [ 33 ] 34 35 36 37 38 39 40 41 42 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved