Forum Moderators: open
Crawled some of my stuff, I thought it was a surfer, but I didn't see it come from any engine, and the amount of stuff it grabbed really through me for a loop, especially the regularity of the requessts.
Does any one know if this is a spider, or if some surfer just couldn't get enough of my redirect system, and had to click on everything?
Thanks,
han solo
216.160.0.0 - 216.161.255.255
U S WEST Interact Services
600 Stinson Blvd NE
Minneapolis, MN 55413
us
U S WEST ISOps abuse@uswest.net
612-664-4689
NS1.USWEST.NET 204.147.80.5
NS2.DNVR.USWEST.NET 206.196.128.1
USW-INTERACT98
Updated: 18-Jul-2000
Source: whois.arin.net
Of course you already knew this.
Does it usually mean when the host is unreachable that it most likely is a dial-up account?
The reason I posted this was the activity on my logs...it seemed more like a spider than a surfer, and I handed it the information I give to web surfers, and not to spiders...
What I've been doing is custom serving different entities, and since this "user" didn't seem to come from a search engine, I got confused, since it seemed like a surfer with the ip/ua combo it gave...but then when it grabbed so much stuff...makes me wonder.
Perhaps the gods of seo will be along shortly to help us...;)
Cheers,
Han Solo
What I was trying to say here is that there is a program which will come to your site and look like a spider. It gives no UA (if you don't want it to) and no referrer. The only way to determine whether it is a search engine or a surfer running his own bot is by ip. As long as you are cloaking by ip delivery, you are OK.
Edited by: eljefe3
With some of the engines, surfers doing odd things, I've begun to get very paranoid...after a while, if anything comes through and looks even slightly abnormal, I spend at least an hour tracking it down, checking the ip, and trying to assure myself I didn't just redirect the latest version of slurp :)
Thanks again for your help.
Cheers,
Han Solo
