homepage Welcome to WebmasterWorld Guest from 50.19.199.154
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconPrevent spider from following external links
  to keep Amazon items out of search engine database
2Reno 7:11 am on Aug 25, 2006 (gmt 0)
newpost thread iconRobots txt to remove pages
  Does google still look at urls blocked
3cheaperholidays 7:56 am on Aug 24, 2006 (gmt 0)
newpost thread iconWhere do I find the Robot Validation Tool?
  robot validator validation robots
7myebux 9:45 pm on Aug 23, 2006 (gmt 0)
newpost thread iconNeed advice on my robots.txt file
  new to robots.txt, can someone check this file
4jwa55121 5:07 pm on Aug 22, 2006 (gmt 0)
newpost thread iconAre there any tricky uses of robot.txt file?
  robot.xt file to be indexed by SEs?
4yuvika 8:47 am on Aug 22, 2006 (gmt 0)
newpost thread iconNew spider that is actually good.
  found a "good" new spider
1jcrimi 12:10 am on Aug 22, 2006 (gmt 0)
newpost thread iconrobot.txt
  problem disallowing pages in home directory
2surfpro203 2:23 am on Aug 16, 2006 (gmt 0)
newpost thread iconFlash web site - need for robots.txt?
  I have a web site in Flash - is it necessary to have a robots.txt file?
2Cratima 1:05 pm on Aug 15, 2006 (gmt 0)
newpost thread iconHttps? Where'd that come from?
  Site dropped like a rock
2Dragonlady26 1:56 am on Aug 12, 2006 (gmt 0)
newpost thread iconPlease double check my robots.txt
 
1lee_sufc 9:30 am on Aug 7, 2006 (gmt 0)
newpost thread iconRobots.txt necessary
  Is the robots.txt file absolutely necessary
5camchoice 6:57 pm on Aug 6, 2006 (gmt 0)
newpost thread icon'robot.txt' file to be indexed by SEs?!
  are there any tricky uses of robot.txt file?
2yuvika 6:56 pm on Aug 6, 2006 (gmt 0)
newpost thread iconDisallow index.html in Robots.txt
  Is it safe to disallow an index.html file of a website
3pixeladdict 8:48 pm on Aug 4, 2006 (gmt 0)
newpost thread icondisallow every 'filename' except.
  disallow every 'filename' except...
2decbrad 4:12 am on Aug 4, 2006 (gmt 0)
newpost thread iconRobots.txt Exclusion vs. Nofollow Tag
 
3JasperSEO 1:39 am on Aug 1, 2006 (gmt 0)
newpost thread iconKeep robots away from site at least two weeks
 
1zoth 2:43 pm on Jul 29, 2006 (gmt 0)
newpost thread iconRobots.txt exclusion help
  Robots.txt exclusion help
3webguy7 4:57 am on Jul 22, 2006 (gmt 0)
newpost thread iconusing robots.txt
  is it necessary to exclude .php pages
6deborahbaker 11:53 am on Jul 13, 2006 (gmt 0)
newpost thread iconRobots exclusion revisted
  How far have we come with controlling robots?
3pixel_juice 11:12 pm on Jul 7, 2006 (gmt 0)
newpost thread iconNeed to redirect bots from subdomain to main site
 
1nemo1208 3:28 pm on Jul 7, 2006 (gmt 0)
newpost thread iconHelp with Spiders
  Cutting back on the bandwidth
2devnet 4:56 am on Jul 5, 2006 (gmt 0)
newpost thread iconIs there any harm in not having a robots.txt file at all?
 
21Lit 10:55 am on Jul 4, 2006 (gmt 0)
newpost thread icon90% of my traffic is spiders and bots.
  What to do -
4webpro00801 6:41 am on Jul 3, 2006 (gmt 0)
newpost thread icontwo duplicate sites .com .net
  confused on how to handle mirror sites
4jcnh74 9:33 pm on Jul 1, 2006 (gmt 0)
newpost thread iconGoogle 5000 character limit for robots.txt?
 
6physics 8:53 pm on Jul 1, 2006 (gmt 0)
Displaying Page 35 of 71<< Back 1 ... 25 26 27 28 29 30 31 32 33 34 [ 35 ] 36 37 38 39 40 41 42 43 44 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved