Yahoo now spidering dynamic sites

But they leave the shopping cart id in messing up my shopping carts.

[lgn1]

We have a dynamic site and in the past this was not
scanned. I see that Yahoo is now scanning dynamic sites but now I have a problem.

They are picking up url's such as:

www.example.com/cgi-bin/webscript.pl?page=catalog224c.html&cart_id=123456.789

They are actually picking up the unique shopping cart id instead of

www.example.com/cgi-bin/webscript.pl?page=catalog224c.html&cart_id=

which would force the generation of a new cart id.

Right now, mulitple users are using the same shopping cart, with chaotic results.

Is there anyway to force the spider to skip the unique shopping cart id and generate results in the form:

www.example.com/cgi-bin/webscript.pl?page=catalog224c.html&cart_id=

[enotalone]

I would modify the code so it checks for user agent and hides/does not generate cart_id values if sees it is a robot.

[lgn1]

My understanding it that every search engine has its own robot string and its subject to change, which could be a logistics nightmare, or am I missing something.

Please clarify.

[enotalone]

You are right lgn1, every search engine has its own user agent. What you could do at least prepare your code for the major ones such as google, yahoo, ask. they hardly ever change their user agents, you will not need to update that list more than once a year or so.

i don’t know how else you could fix that problem other than modifying your code so it can help SEs crawl properly.

i had similar but not that serious problem with my forum (phpbb) pages sometimes being indexed with session ids and that is the way i fixed it and works fine for the last year or so.