homepage Welcome to WebmasterWorld Guest from 54.198.224.121
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDisallow by ISP
  Where to get a list of ISPs and the countries they are in?
1youfoundjake 7:13 pm on Nov 3, 2006 (gmt 0)
newpost thread iconMade the mistake of putting sub folders I want allowed
  disallow everthing but?
2Sunnyvale 7:07 pm on Oct 31, 2006 (gmt 0)
newpost thread iconGoogle remove url tool for supplemental urls a waste of time?
  Should I bother removing duplicate supplementals?
3Tonerman 4:20 pm on Oct 30, 2006 (gmt 0)
newpost thread iconallow major engines, block the fleas
  and to add further control
5youfoundjake 5:16 pm on Oct 26, 2006 (gmt 0)
newpost thread iconDissallowing Dynamic Content
  Blocking "index.php?"
2trillianjedi 6:58 pm on Oct 25, 2006 (gmt 0)
newpost thread iconCombining htaccess and robots.txt?
 
7Lady18 11:37 pm on Oct 24, 2006 (gmt 0)
newpost thread icondisallowing a 'staging server'
 
2rden17 4:11 pm on Oct 24, 2006 (gmt 0)
newpost thread icon.htaccess
 
2ssgumby 9:53 pm on Oct 21, 2006 (gmt 0)
newpost thread iconcloaking robots.txt
  to avoid hacking
4idolw 12:26 pm on Oct 18, 2006 (gmt 0)
newpost thread iconHow to deal with a robots.txt hack attack
  Guidelines to mitigate the damage from an attack
3Whitey 10:17 pm on Oct 17, 2006 (gmt 0)
newpost thread iconMainseek bot
  this bot is ripping through our site
2ssgumby 6:36 pm on Oct 14, 2006 (gmt 0)
newpost thread iconGoogle indexing weird links
  ones with // in them
4netchicken1 12:54 am on Oct 13, 2006 (gmt 0)
newpost thread iconPlease welcome your new moderator: goodroi
 
20engine 6:35 am on Oct 11, 2006 (gmt 0)
newpost thread iconrobots.txt problem and subdirs / wildcards
 
2disgust 1:44 pm on Oct 10, 2006 (gmt 0)
newpost thread iconGoogle Sitemaps Robots.txt Question
 
2Tonerman 5:31 pm on Oct 9, 2006 (gmt 0)
newpost thread iconNaming da' bots
  Same place, different face
1Reno 11:58 am on Oct 7, 2006 (gmt 0)
newpost thread iconTravelLazerBot
  Anyone seen this bot before?
2jexx 7:44 pm on Oct 3, 2006 (gmt 0)
newpost thread iconDisallow Coldfusion Pages Only
  Stop cfm pages from being crawled
2stcrim 3:18 pm on Oct 3, 2006 (gmt 0)
newpost thread iconRemoving specific file type
 
8MisterT 11:40 pm on Sep 28, 2006 (gmt 0)
newpost thread iconRobots - Allowing them
  rather than disallow
2SanDiegoPaul 10:30 pm on Sep 28, 2006 (gmt 0)
newpost thread iconRobots.txt & meta
  Robots meta tage or just a robots.txt
4pazang 7:45 pm on Sep 27, 2006 (gmt 0)
newpost thread iconDo excluded pages still count as links-in
 
3johnlod 8:14 pm on Sep 26, 2006 (gmt 0)
newpost thread iconOrder of lines in robots.txt file
  Can you control a bot with the order of the lines?
3bouncybunny 3:37 am on Sep 23, 2006 (gmt 0)
newpost thread iconrobots & blog
 
2zuses 2:54 pm on Sep 22, 2006 (gmt 0)
newpost thread iconKeeping robots out of cgi bin
  What's the best format?
10Reno 6:00 pm on Sep 21, 2006 (gmt 0)
Displaying Page 33 of 71<< Back 1 ... 23 24 25 26 27 28 29 30 31 32 [ 33 ] 34 35 36 37 38 39 40 41 42 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved