Borek - 1:00 pm on Aug 21, 2005 (gmt 0)

On my site I have a script generating dynamically some funny/stupid content (recurence - see recurence type joke). URLs take form example.com/something.cgi?cacbbabc and so on. Up to August 8th Googlebot have not tried to spider the script. Since August 8th it started to crawl deeper and deeper, to the point I was forced to modify the script, to force end of crawling.

Perhaps Gbot code was modified and it is able to crawl dynamically generated pages much better then before? IIRC phpbb forums using ID's in the url were not spidered up to now (or at least were not always spidered - I know pretty old sites with high PR, no robots.txt, no nofollow tags, where phpbb is used and forum is not indexed).

Best,
Borek