Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Beginners guide to robots


Monkscuba - 10:43 am on Sep 29, 2002 (gmt 0)


Dear webmasters,

As a complete amateur, our web site may look a bit shabby, but right now we can't afford fancy web design. However, largely thanks to this Wemaster site, we get high rankings for our keywords, and I am still trying to optimise. The site has only been online since June, and I am gathering links and hopefully customers!

Anyway, I am very confused about all this robots.txt talk, and want to know if we have one or not! NO and we don't need one!I think it's the bit which says :

<meta name="robots" content="index,follow">

Yes? NO! That's the robots meta tag. Wow, you learn more every day!

If I want to disallow robots, where does this code go? Under the previous line? Is it worth disallowing robots?

Any pros got any comments on our site? Am I doing anything really wrong? Our market is very competitive on the ground, but most of the competing websites are not so great, and a PR of 5 is about as high as you'll find in our field (we are currently PR 4).

Sorry to bother y'all like this.

Have a nice day!

Hey! I can't see my homepage on my profile! How can anyone help me!

[edited by: Monkscuba at 1:26 pm (utc) on Sep. 29, 2002]


Thread source:: http://www.webmasterworld.com/robots_txt/173.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com