ASP pages and crawlers!

I just want to re-raise one issue, about dynamically-generated pages.

[anubis]

I just want to re-raise one issue, about dynamically-generated pages. 
The point about using search engine crawlers not recognising parameter strings
(after the '?' in a URL) is understood should be avoided.  However, I'm not sure why .ASP pages *in
themselves* (i.e. without a parameter) are a problem.

As far as I know the only way that a crawler
can distinguish between a .ASP and a .HTML file is through the file type.
(The HTTP Headers of each return the same Content-Type).  And it looks to be very easy for a webmaster to configure the server to generate ASP pages with
a .HTML file type - i.e generate the page dynamically, but give the page a .HTML file type.

Give that, and the ease with which you could sidestep any penalisation of .ASP over .HTML, can I ask you guys to just confirm that I should avoid non-parameterised .ASP pages?  And if so, can I get round it by configuring
the server to serve up Active Server pages with .HTML extensions?

Cheers

[Dreamquick]

Who said straight ASP pages (ie not doing any fancy redirects based on cookies/sessions and without any querystring) are a problem?

What specifically do you think is the problem with them?

As far as I know unless you start doing things with server-side code which search engines can't cope with (ie forcing session ids to be used) or make them "nervous" about accessing pages (ie putting lots of paramters / long strings into the querstring) then you shouldn't have any problems.

To quote the googleguy from a recent discussion;

General rule of thumb is that Googlebot is willing to ingest just about anything. The corollary is to keep the number of parameters small and to keep those parameters short (no session IDs, for example).

(from [webmasterworld.com...]

I (plus various other people who normally chime in) have no problem getting crawlers to hit *our* ASP based sites and stay for a while - I had a look at a previous months logs the other day and Googlebot had requested a fair few pages (I know it was into several hundreds 100's)...

- Tony

[edited by: Dreamquick at 4:32 pm (utc) on Jan. 10, 2003]

[DaveN]

anubis i agree with Dreamquick

check your sticky

DaveN

[chrisforeman2]

Almost all of my work is in ASP and have no problem with indexing, even pages with stuff like xxx.asp?id=21 work just fine.

It may be possible that very long session strings may be a problem as they include temp stuff (which a session is)

As far as I know if your asp page has extra stuff on the end that takes it to a page it works fine, it just gets strange if you have temp stuff on the end like sessions info

chris