Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Blocking Includes (SSI)


direktor1 - 2:22 pm on Jan 7, 2004 (gmt 0)


Hi,

I'm having trouble using server-side includes for a drop down menu on my site. When the robot traverses a page it reads everything in the drop down menu and since the menu is on every page, it lowers the relevancy of search results.

If I all a line:
Disallow: /include.inc

Or

Disallow: /includes/

to my robot.txt file, will that block the robot looking at the inluded file on the page or will it still read the inlcude after the code has been processed.

If that does not work, can anyone suggest a way to get the robot to not look at a drop down menu in an include other than using client-side scripting.

Thanks

Adrian


Thread source:: http://www.webmasterworld.com/robots_txt/230.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com