Included page - search engine issues

Is included page indexed by search engines?

[Chris27]

I am using the Include page feature.

I have been planning to put the 'sub-pages' that the include statement refers to in the private folder, since if found by search engines, they would not make much sense, I would want the visitor to see te composite page.

My concern is that I would want the text and other content pulled from the sub-page to be included in the google crawl and to be indexed.

For example my HTML may have the following:

<!--webbot bot="Include" U-Include="_private/gardens.htm" TAG="BODY" -->

If search engines do not consider the text that is in gardens.htm, the content included in it would miss out on being indexed.

Do search engines piece together the completed page so that I am OK?

thanks for your help

regards

Chris

[webboy1]

I believe SE's do pick up all info contained in includes. We use includes quite a lot with our ASP for many reasons - some technical and some simply as timesavers for future site updates.

To date, we have no reason to believe that our includes are not being crawled. In fact, our main navigation on several of our sites is contained within include files. If includes were not crawled then the SE's would never have found the internal pages of our sites ..... which they have.

Hope this helps!

[Chris27]

Thanks Webboy,

do you keep your included pages in the private folder?

regards

Chris

[webboy1]

Hi Chris27,

I keep them in a sub-directory called 'includes'. Like you, i don't want this directory crawled as if they were individual pages, so i simply declare in my robots.txt that this folder should be ignored i.e.:

User-agent: *
Disallow: /includes

This leaves the spiders to read the include contents through the actual page they are being included in - and not as individual files.

I am not sure if this is the perfect solution, but i can safely say that to my knowledge, NONE of my individual include files have appeared on any SE, but that their content HAS been read via the page they are included in.

[Iguana]

Includes are processed on the server side - the browser/bot never sees the include declaration - just the result.

Call the resulting page up in a browser - view source - that's what people/bots see.

[pageoneresults]

The spider will see whatever you see when viewing the source of a web page at the browser level.

Also, the _private folder in FP is inaccessible by default and should not be used for anything other than stuff that absolutely cannot be indexed such as form results, etc. You can put your includes in a sub called /includes/ as mentioned above.

Includes are processed on the server side - the browser/bot never sees the include declaration - just the result.

In FP this is a little different. If the developer is using .htm or .html extensions, the include statement is visible. You will see a beginning (startspan) and ending (endspan)

<webbot>

tag indicating included content.