Welcome to WebmasterWorld Guest from 18.104.22.168
Crawled some of my stuff, I thought it was a surfer, but I didn't see it come from any engine, and the amount of stuff it grabbed really through me for a loop, especially the regularity of the requessts.
Does any one know if this is a spider, or if some surfer just couldn't get enough of my redirect system, and had to click on everything?
22.214.171.124 - 126.96.36.199
U S WEST Interact Services
600 Stinson Blvd NE
Minneapolis, MN 55413
U S WEST ISOps firstname.lastname@example.org
Of course you already knew this.
Does it usually mean when the host is unreachable that it most likely is a dial-up account?
The reason I posted this was the activity on my logs...it seemed more like a spider than a surfer, and I handed it the information I give to web surfers, and not to spiders...
What I've been doing is custom serving different entities, and since this "user" didn't seem to come from a search engine, I got confused, since it seemed like a surfer with the ip/ua combo it gave...but then when it grabbed so much stuff...makes me wonder.
Perhaps the gods of seo will be along shortly to help us...;)
What I was trying to say here is that there is a program which will come to your site and look like a spider. It gives no UA (if you don't want it to) and no referrer. The only way to determine whether it is a search engine or a surfer running his own bot is by ip. As long as you are cloaking by ip delivery, you are OK.
With some of the engines, surfers doing odd things, I've begun to get very paranoid...after a while, if anything comes through and looks even slightly abnormal, I spend at least an hour tracking it down, checking the ip, and trying to assure myself I didn't just redirect the latest version of slurp :)
Thanks again for your help.