Forum Moderators: open

Message Too Old, No Replies

Yahoo requesting GET vars that don't exist

It appears that slurp is auto-incrementing a get variable

         

stargeek

8:14 pm on Sep 22, 2005 (gmt 0)

10+ Year Member



I've never seen this before but it appears that Slurp is spidering and indexing dynamic urls that don't exist like this:

page.php?IDnumber=1
page.php?IDnumber=2
page.php?IDnumber=3
page.php?IDnumber=4
page.php?IDnumber=5

and so on, these id's don't exist in the database and are never linked to from the site. They used to return blank pages, but we're making a fix to send a 404 on these pages.

has anyone seen this before?

PS neither google nor msn are doing this either.

stargeek

3:40 pm on Sep 26, 2005 (gmt 0)

10+ Year Member



So no-one else has seen this auto incrementation spidering?

Dijkgraaf

12:36 am on Sep 27, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'd throw a 400 or a 410 rather than a 404, as most bots will re-request a page after getting a 404 on it previously.