Dynamic Pages in AV

Anyone else?

[sugarkane]

Well, I've finally got around to looking into why one site as been buried for the last couple of weeks. Seems that they've indexed 92 variations of our shopping cart page, by following cart.cgi?prod_code links. All those near-identical pages could well result in a burying.

Anyone else found this?

BTW - Just checked and it's happened on the majority of my sites using that kind of shopping cart.

[rcjordan]

I've recently seen a few AV referrals on a dynamic site I have... but this site is cached by my server to reduce the cpu load. Some 5 or 6 months ago it turned up in INK referrals and made it appear as if INK was indexing dynamic pages, we finally figured out it was reading the cached versions. Any chance of that happening on your site?

[sugarkane]

No, I don't think so. Even if the server was cacheing, doing a url:www.mydomain.com/scripts/ search brings up over 100 pages with cgi?p=1&ref=foo type URLs. All pages are accessible only by following dynamic links.

Interestingly, I've checked every listing and there's one there that was only on the server for a matter of weeks over a year ago, and only accessible by following a dynamic link from within the cart itself (ie 2 levels deep into dynamic pages).

Do I remember rightly that AV experimented with indexing dynamic stuff around that time? It could be a blast from the past...

[seth_wilde]

"AV experimented with indexing dynamic"

This is interesting, AV has indexed dynamic pages if they were directly submitted for quite some time, but this is the first I've heard of them crawling dynamic links.

Anybody else experience anything similar?

[oLeon]

AV has crawled dynamic pages. As I remember right, these results are from the beginning of 2000. Try that search [altavista.com]. There are a lot with different "pWkorbID", i.e. the ID of every single "basket of goods".