Forum Moderators: open

Message Too Old, No Replies

Hello Crawler

         

keyplyr

11:19 pm on Sep 1, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




UA: Hello Crawler
Protocol: HTTP/1.1
Robots.txt: No
Host: plus.net
146.200.0.0 - 146.200.255.255
146.200.0.0/16

dstiles

7:17 pm on Sep 2, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Actually a UK BT service via INFONET running in USA on 146.198.0.0/14. I have it listed as broadband with only 5 minor baddies across the range since I listed it April this year.

keyplyr

8:25 pm on Sep 2, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@dstiles - Thanks for that info. I'm not quite sure what Hello Crawler is purposed for, but it gets blocked by default until it provides info that it is useful to my interests.

blend27

2:41 am on Sep 3, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@keyplyr

Aren't you(or were) running your own crawler? Maybe they are just saying Hello? :) or just up to you?....

What did You doooooo?

keyplyr

3:20 am on Sep 3, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@blend27 - so it is saying hello to my crawler? I get it :)

However the bot hit a site I work on, not my own. Most bad bots don't get through to my own site. I usually see these UAs in logs from other servers.

<off topic>
It amazes me how little defense some online companies have when it is their very business that's at stake. The business owners have no idea what is happening on the internet. Then they get scraped/copied or hacked and their branding, consumer trust & traffic is lost and they don't understand how it happened.
</off topic>

blend27

1:36 pm on Sep 3, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



<ot>
Then they get scraped/copied or hacked

I've recently picked up a few smaller side projects from following logs of one of the my new clients(their site was hacked). As soon as you show them what Google brings up what you do a "site:" command, they open their wallets and start paying real close attention. Mostly WordPress and Magento older spaghetti code plugins.

So far 2 of those projects turned into a several thousand dollars in billable hours by offering these pips a new way of thinking about their online presence.

;)
</ot>