Forum Moderators: open

Message Too Old, No Replies

BLEXBotTest

         

keyplyr

10:43 am on Apr 28, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: Mozilla/5.0 (compatible; BLEXBotTest/1.0; +http://webmeup.com/crawler.html)
Protocol: HTTP/1.1
Robots.txt: Yes
Host: BelPrime Solutions (belprime.com)
216.176.185.0 - 216.176.185.31
216.176.185.0/27

Also seen from: Mobile TeleSystems (mts.by)
134.17.16.0 - 134.17.31.255
134.17.24.0/21

Marketing, SEO & Backlinks report service. Also seen as "BLEXBot"

lucy24

7:20 pm on Apr 28, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Also seen from

198.143.132.146
2014 through early 2015, overlapping with
136.243.36.82
which in turn overlaps with the 134.17. above. More recently (early this year) seen from
148.251.3.137
and then on to
144.76.107.164
(Exact IP each time, over several visits.)

They move around a lot, don't they? But it all appears to be the same robot. Unlike some robots one could name, the robots.txt request comes a second or two before the page request. (I've never understood the ones that do it the other way around.)

keyplyr

11:08 pm on Apr 28, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I allow this UA but hope they get their crawl tactics in better order. Maybe this "Test" version is purposed for doing that. Still, they do need a dedicated crawl range.