Welcome to WebmasterWorld Guest from 54.205.20.160

Forum Moderators: goodroi

Message Too Old, No Replies

SEO Robot.txt tag and robot meta tags... need advice

Help about robot.txt and robot meta tags

   
5:15 pm on Jan 25, 2004 (gmt 0)

10+ Year Member



Hey Guys,

I own the site: www.example.co.uk and I'm setting it up for the Search Engines.

First question - should you ban spiders indexing contact forms and/or the thank you for submitting a form pages?

At the moment I'm using a robot.txt file, check the test page: www.example.co.uk/robots.txt

Should I use a robot.txt or robot meta tags or both to ban indexing of certian pages.

Final question - should you ban spiders from your image files?

Thanks

Jamie

5:23 pm on Jan 25, 2004 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



First, welcome to WebmasterWorld.

>Should I use a robot.txt or robot meta tags

It's a toss up. Whichever is easier to setup and mantain. I think the real question is how many files you are talking?

> should you ban spiders from your image files?

I think so. However, there are some sites that seel product based on those images - so they would want to allow them.

Go2

2:09 pm on Jan 28, 2004 (gmt 0)

10+ Year Member



Should I use a robot.txt or robot meta tags or both to ban indexing of certian pages.

I agree with Brett, go with what is easiest to maintain. If you have many web pages on your site and the web pages have been organized in file directories, it may be easier to use robots.txt.

On a side note, if you choose to use both ways, and there is a conflict between the directive in robots.txt and the meta tag directive (e.g. if your robots.txt allows spidering but your page meta tag does not) the robots.txt directive has precedence.