Will a crawler index this?

Forum Moderators: phranque

Message Too Old, No Replies

Will a crawler index this?

dbdev

1:03 pm on Jul 13, 2003 (gmt 0)

I do not want crawlers to index some parts of my page content. If (reference the below example) the file content-i-dont-want-indexed.asp resides in a directory that is restricted by the robots.txt "noindex/nofollow" - will the crawler index the rest of the page and just leave out the content-i-dont-want-indexed.asp content?


<body>
some content....
<!-- 
<%#include file="/donotindex/content-i-dont-want-indexed.asp"%>
-->
more content....
</body>

Is there a better way to do this? I thought about using java script to include the file because crawlers cant read java script right?

I am new to seo and any help would be greatly appreciated.

Regards,
Michael Crawford.

PS: The copy I dont want crawlers to index is made up of examples for a certain topic I am discussing - not trying to pull any kind of funny business on search engines or anything like that.

tschild

1:37 pm on Jul 13, 2003 (gmt 0)

If I understand correctly your solution includes the content of content-i-dont-want-indexed.asp in the complete page via a server-side mechanism. In that cas the spider would get it as part of the whole page - no way for the spider to know whether the content was from the page code or included from a location where the sider may not go.

Using Javascript would exclude quite a few visitors.

What about including the data via <iframe> or <object>?

dbdev

8:11 pm on Jul 14, 2003 (gmt 0)

Thanks for the speedy reply,

You mention that "Using Javascript would exclude quite a few visitors." Why is this?

Your

 <iframe>

suggestion is a good idea (not sure how to use the

<object>

Thankyou
Mike.

cyt0plasm

8:25 am on Jul 16, 2003 (gmt 0)

Requiring Javascript would exclude a lot of users because, like me, many people browse with it off. Some don't support it, some gets mangled by ad removal proxies (privoxy, admuncher), and some times, the javascript is just broke (although, a simple document.write isn't likely to fall into that category).

dbdev

12:34 pm on Jul 19, 2003 (gmt 0)

The only thing I am using javascript for is to toggle the visibility state of a

<div>

tag through a hyperlink, and a "top bar" menu.
Here is how I am toggleing the visibility of the

<div>

using a hyperlink.


<a href="#" onMouseOver="toggle_visibility('div_name')">
//toggle_visibility() is a javascript function located at the bottom of my html code.

Is there a way to toggle the visibility without javascript? Perhaps using vbscript (browsers don't have a choice in functioning properly with vbscript?) -> is this true? If so how can I do it with vbscript?

I can take care of the javascript "top bar" menu by simply replacing it with hyperlinks (not as pretty though).

Your advice is most appreciated,
Michael Crawford.