Forum Moderators: open

Message Too Old, No Replies

Unknown bot scrape or probe?

         

enigma1

10:23 am on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



188.40.82.nnn - - [16/May/2010:04:51:17 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:18 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:18 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:18 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:18 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"
188.40.82.nnn - - [16/May/2010:04:51:19 -0400] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.15) Gecko/2009102100 SUSE/3.0.15-0.1.2 Firefox/3.0.15"

Multiple hits within a couple of seconds, it happens frequently nothing new but what I found strange are the Additional Records derived from the IP.

Additional-1
host: ASPMX.L.GOOGLE.COM
type: A
ip: 209.85.221.nnn
class: IN
ttl: 54
Additional-2
host: ALT1.ASPMX.L.GOOGLE.COM
type: A
ip: 216.239.59.nnn
class: IN
ttl: 70
Additional-3
host: ALT2.ASPMX.L.GOOGLE.COM
type: A
ip: 74.125.79.nnn
class: IN
ttl: 70
Additional-4
host: ASPMX2.GOOGLEMAIL.COM
type: A
ip: 209.85.135.nnn
class: IN
ttl: 1829
Additional-5
host: ASPMX3.GOOGLEMAIL.COM
type: A
ip: 72.14.213.nnn
class: IN
ttl: 1829

The IP itself belongs to some host in Germany.

incrediBILL

6:14 pm on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Do a little more WHOIS work, the googlemail.com and other google.com addresses and IPs you provide actually resolved to Google IP blocks.

dstiles

8:21 pm on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



188.40.82.nnn is Hetzner servers in Germany - I get wuite a few hits from their servers with a variety of bot and speudo-browser tags.

How did you resolve 188.40.82.nnn to the google records?

enigma1

8:34 pm on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I just requested the DNS resource records for a given hostname. So first you can check if the IP->Host->IP is valid, then you request the DNS records with PHP that I use there is a ready function dns_get_record that returns all kinds of DNS info.

@Bill so have you previously experienced that? I do not often check every record aside from the host but happened in this case.

caribguy

8:58 pm on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What you're seeing are MX records. That server is probably using Google Apps / GMail for hosted domains.

And, yep... Hetzner is notorious for "interesting" traffic.

enigma1

8:05 am on May 18, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes I noticed the mail servers and was wondering about...
That server is probably using Google Apps / GMail for hosted domains.

..this, it makes sense now, so they don't have to worry about mail services or a mail server they can use mail by another provider.

Of course this could work in favor of the webmasters because they can identify such "features" and allow/block access for the IP.