Welcome to WebmasterWorld Guest from 54.226.246.160

Forum Moderators: goodroi

Message Too Old, No Replies

meta name = robot

What does that do?

     

aroye

8:44 pm on Jun 29, 2003 (gmt 0)

10+ Year Member



What exactly does <meta name="robots" content="index, follow"> do? and how does it differ from robots.txt?

Symbios

8:46 pm on Jun 29, 2003 (gmt 0)

10+ Year Member



In my opinion nothing, the robots will follow the links on your webpage whether you have this tag or not.

The robots.txt file will tell the robots which files you wish to restrict access to.

Dreamquick

8:53 pm on Jun 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



For all the info on the tag have a look here;

[searchengineworld.com...]

As for how it differs to robots.txt well...

Robots.txt is at a site level and simply acts as a signpost saying "please don't crawl these" and list of paths, however the robots meta-tag allows you to be more granular - for instance "index,nocrawl" would tell them "index this page but don't crawl any of the links that it uses" - something which can't be done with robots.txt.

Oh and welcome to WebmasterWorld [webmasterworld.com] aroye!

- Tony

...and if I do say so myself a fitting way to hit my 500th post.

RonPK

8:54 pm on Jun 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi aroye, welcome to WebmasterWorld!

In addition to Symbios' reaction: here is some more or less official documentation: [robotstxt.org...]

Symbios

8:56 pm on Jun 29, 2003 (gmt 0)

10+ Year Member



Dreamquick,

Very fitting and very useful. I'm looking forward to my 50th post.

aroye

9:05 pm on Jun 29, 2003 (gmt 0)

10+ Year Member



Thanks Guys!

jdMorgan

5:57 pm on Jun 30, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Some readers of this thread might find this thread [webmasterworld.com] interesting. If you want to control the "presentation" of your pages in search engine results, there are very good reasons to use a mixture of both robots.txt and the robots meta tag.

Jim

 

Featured Threads

Hot Threads This Week

Hot Threads This Month