Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google Index robots.txt


webseosolution - 1:08 pm on Aug 24, 2012 (gmt 0)


Hello,

Let me explain my question in depth.

Let's say on my URL [xyz.com...] page having following content in page source

<meta name="robots" content="all" />
<meta name="robots" content="index, follow" />
<meta name="googlebot" content="index, follow" />
<meta name="msnbot" content="index, follow" />

Now I am setting following rule in .htaccess file

<files admin.php>
Header set X-Robots-Tag "noindex, nofollow"
</files>

And my admin.php is NOT restricted in robots.txt

Question: How bot will behave in above mentioned case?

Thanks & Regards,
WSS


Thread source:: http://www.webmasterworld.com/robots_txt/4487907.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com