Welcome to WebmasterWorld Guest from 188.8.131.52
one frame in this page is /dir1/bottom.htm and in my robots.txt file I disallow googlebot from /dir1/
does /page/index.htm get indexed?
<meta name="robots" content="noindex">
JamesR is right. SE looks at individual pages called by framesets as what they are, individual pages.
Any good stuff for se spiders to eat in your /page/index.htm page?
Most se will abide by <meta name="robots" content="noindex"> command.
/page/index.htm is disallowed by both of these cases I was wondering if googlebot was as stupid as slurp
Become a Pro Member