homepage Welcome to WebmasterWorld Guest from 54.226.161.112
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconAre there any tricky uses of robot.txt file?
  robot.xt file to be indexed by SEs?
4yuvika 8:47 am on Aug 22, 2006 (gmt 0)
newpost thread iconNew spider that is actually good.
  found a "good" new spider
1jcrimi 12:10 am on Aug 22, 2006 (gmt 0)
newpost thread iconrobot.txt
  problem disallowing pages in home directory
2surfpro203 2:23 am on Aug 16, 2006 (gmt 0)
newpost thread iconFlash web site - need for robots.txt?
  I have a web site in Flash - is it necessary to have a robots.txt file?
2Cratima 1:05 pm on Aug 15, 2006 (gmt 0)
newpost thread iconHttps? Where'd that come from?
  Site dropped like a rock
2Dragonlady26 1:56 am on Aug 12, 2006 (gmt 0)
newpost thread iconPlease double check my robots.txt
 
1lee_sufc 9:30 am on Aug 7, 2006 (gmt 0)
newpost thread iconRobots.txt necessary
  Is the robots.txt file absolutely necessary
5camchoice 6:57 pm on Aug 6, 2006 (gmt 0)
newpost thread icon'robot.txt' file to be indexed by SEs?!
  are there any tricky uses of robot.txt file?
2yuvika 6:56 pm on Aug 6, 2006 (gmt 0)
newpost thread iconDisallow index.html in Robots.txt
  Is it safe to disallow an index.html file of a website
3pixeladdict 8:48 pm on Aug 4, 2006 (gmt 0)
newpost thread icondisallow every 'filename' except.
  disallow every 'filename' except...
2decbrad 4:12 am on Aug 4, 2006 (gmt 0)
newpost thread iconRobots.txt Exclusion vs. Nofollow Tag
 
3JasperSEO 1:39 am on Aug 1, 2006 (gmt 0)
newpost thread iconKeep robots away from site at least two weeks
 
1zoth 2:43 pm on Jul 29, 2006 (gmt 0)
newpost thread iconRobots.txt exclusion help
  Robots.txt exclusion help
3webguy7 4:57 am on Jul 22, 2006 (gmt 0)
newpost thread iconusing robots.txt
  is it necessary to exclude .php pages
6deborahbaker 11:53 am on Jul 13, 2006 (gmt 0)
newpost thread iconRobots exclusion revisted
  How far have we come with controlling robots?
3pixel_juice 11:12 pm on Jul 7, 2006 (gmt 0)
newpost thread iconNeed to redirect bots from subdomain to main site
 
1nemo1208 3:28 pm on Jul 7, 2006 (gmt 0)
newpost thread iconHelp with Spiders
  Cutting back on the bandwidth
2devnet 4:56 am on Jul 5, 2006 (gmt 0)
newpost thread iconIs there any harm in not having a robots.txt file at all?
 
21Lit 10:55 am on Jul 4, 2006 (gmt 0)
newpost thread icon90% of my traffic is spiders and bots.
  What to do -
4webpro00801 6:41 am on Jul 3, 2006 (gmt 0)
newpost thread icontwo duplicate sites .com .net
  confused on how to handle mirror sites
4jcnh74 9:33 pm on Jul 1, 2006 (gmt 0)
newpost thread iconGoogle 5000 character limit for robots.txt?
 
6physics 8:53 pm on Jul 1, 2006 (gmt 0)
newpost thread iconhow to block all robots except for 2?
  possible?
7ionchannels 12:30 pm on Jun 29, 2006 (gmt 0)
newpost thread iconRe-Indexing Timetable
  Any guesses as to when the SE's will display our listings again.
4kgormat 7:42 pm on Jun 24, 2006 (gmt 0)
newpost thread iconDisallow: /#example.html#
  How would the following be viewed by a spider?
4kgormat 3:29 pm on Jun 24, 2006 (gmt 0)
newpost thread iconRobot behaviour
  using the robots meta correctly
1Skylo 12:47 pm on Jun 20, 2006 (gmt 0)
Displaying Page 35 of 71<< Back 1 ... 25 26 27 28 29 30 31 32 33 34 [ 35 ] 36 37 38 39 40 41 42 43 44 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved