homepage Welcome to WebmasterWorld Guest from 54.163.72.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
SiteSpeedBot and misinformation, bad bot
incrediBILL




msg:4262494
 11:33 pm on Feb 3, 2011 (gmt 0)

SiteSpeedBot claims that name is their user agent yet that would never pass my filters.

Yet somehow my data, that shouldn't be there if everything they said is true, magically appears on their servers.

User-agent: SiteSpeedBot
Disallow: /
[indeep76.com...]

Without wasting any time I decided to see what's up and asked the site to crawl a bogus page.

It didn't ask for robots.txt like it claims, asked for the home page as SiteSpeedBot, then proceeded to ask for the bogus page as Firefox.

74.81.199.25 - "GET / HTTP/1.1" 200 2297 "-" "SiteSpeedBot"

74.208.70.160 - HEAD /bogus.html HTTP/1.1" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060308 Linux Mint/7 (Gloria) Firefox/3.0.11"


Then I tried again with a page it could find, and it sort of asked for robots.txt, not very clever really...

74.208.70.160 - "GET /page.htmlrobots.txt HTTP/1.1" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060308 Linux Mint/7 (Gloria) Firefox/3.0.11"

It actually asked for the page HEAD to verify it existed BEFORE asking for the robots.txt file, huh?

HEAD /page.html
GET /page.htmlrobots.txt

What a mess.

However, they claim to operate from multiple data centers around the world, so I'm just scratching the surface of their IPs it would seem.

Here's another IP and UA they used...

74.208.105.25
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100908 CentOS/3.6-2.el5.centos Firefox/3.6.9 GTB7.1"

It seems they like 1&1 for hosting this thing from different data centers.

Anyone got anything else on this beast?

 

Mokita




msg:4262512
 12:16 am on Feb 4, 2011 (gmt 0)

What bothers me about them, is IF they are providing Webmaster Tools, why is their bot visiting unless you asked it to?

I did a little scratching around, looking at whois and the IPs you provided.

The registrant for indeep76.com is located in Ukraine (that's a red flag for me).

Also, Reverse DNS for 74.208.105.25 is mail. smartviper. com. Put Smartviper . com into your browser and get this:
SmartViper a web service that collects and analyzes any data about domains and keywords they are optimized for.


Reverse DNS for 74.208.70.160 is u15371126. onlinehome-server. com. If you type the last bits into a browser, it redirects to www. 1und1. de (1&1 in Germany)

MxAngel




msg:4333986
 2:19 am on Jul 2, 2011 (gmt 0)

Caught today pretending to be GoogleBot.
Robots.txt: NO

74.81.199.25 - - [01/Jul/2011:17:53:30 -0500] "GET [redacted] HTTP/1.0" 403 877 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; [google.com...]

74.208.105.25

canonical name mail.smartviper.com
addresses 208.76.50.69

Contact:
Deeptowm
BOGDAN ANDRIY ()

Fax:
CHERNOBILSKAYA
11a kv.85
Kiev, P 03179
UA


DNS records
name class type data time to live
mail.smartviper.com IN A 208.76.50.69 1800s (00:30:00)
smartviper.com IN NS dns3.registrar-servers.com 1800s (00:30:00)
smartviper.com IN NS dns2.registrar-servers.com 1800s (00:30:00)
smartviper.com IN NS dns1.registrar-servers.com 1800s (00:30:00)
smartviper.com IN A 208.76.50.66 1800s (00:30:00)
smartviper.com IN NS dns5.registrar-servers.com 1800s (00:30:00)
smartviper.com IN NS dns4.registrar-servers.com 1800s (00:30:00)
smartviper.com IN SOA server: dns1.registrar-servers.com
email: hostmaster.registrar-servers.com
serial: 2008080810
refresh: 10001
retry: 1801
expire: 604801
minimum ttl: 3601
3601s (01:00:01)
smartviper.com IN TXT v=spf1 ip4:208.76.50.69/32 a ~all 1800s (00:30:00)
25.105.208.74.in-addr.arpa IN PTR mail.smartviper.com 86400s (1.00:00:00)

Traceroute
Tracing route to mail.smartviper.com [74.208.105.25]...

hop rtt rtt rtt ip address fully qualified domain name
1 1 1 1 70.84.211.97 61.d3.5446.static.theplanet.com
2 1 1 0 70.87.254.1 po101.dsr01.dllstx5.networklayer.com
3 18 1 1 70.85.127.105 po51.dsr01.dllstx3.networklayer.com
4 1 1 0 70.87.255.25 e4-2.ibr03.dllstx3.networklayer.com
5 1 6 14 64.125.199.93 xe-3-0-0.er1.dfw2.us.above.net
6 1 1 1 64.125.27.73 xe-0-1-0.cr1.dfw2.us.above.net
7 28 28 28 64.125.30.62 xe-2-1-0.cr1.ord2.us.above.net
8 28 28 28 64.125.26.250 xe-1-1-0.er1.ord7.us.above.net
9 29 28 28 206.223.119.24 equinix.bb-b.cr.chi.us.oneandone.net
10 39 39 39 74.208.1.54 te-2-4.bb-d.ws.mkc.us.oneandone.net
11 40 39 39 74.208.1.89 te-1-1.bb-c.slr.lxa.us.oneandone.net
12 40 40 40 74.208.1.117 ae-11.gw-distp-a.slr.lxa.oneandone.net
13 40 40 40 74.208.1.167 ae-1.gw-prtr-r5-a.slr.lxa.oneandone.net
14 40 40 40 74.208.105.25 mail.smartviper.com

Trace complete

Service scan
FTP - 21 Error: TimedOut
SMTP - 25 Error: TimedOut
HTTP - 80 HTTP/1.1 403 Forbidden
Server: nginx
Date: Sat, 02 Jul 2011 10:25:10 GMT
Content-Type: text/html
Content-Length: 162
Connection: close
POP3 - 110 Error: TimedOut
IMAP - 143 Error: TimedOut

From CentralOps
[centralops.net...]


First result in Google for that IP returns:

Hutsler Law Firm
74.81.199.25/ - Cached
Birmingham, Alabama, attorneys at Hutsler Law Firm represent clients in consumer law.

Edit:

The following A records are set to 74.81.199.25:
appointmentnet.com, hutslerlaw.com, hutslerlawfirm.com, localpsychiatrists.com, ourstatesunited.com, ourstatesunited.org

[bgp.he.net...]

lucy24




msg:4334014
 4:08 am on Jul 2, 2011 (gmt 0)

Hutsler [**typo for Hustler?] Law Firm


74.208-209
That's OneandOne, aka 1and1. I've got them flagged as "kinda hinky though I can't say where or how I picked up this idea". (See further up this thread.)

74.80-81 maximumasp
asp = viper, right? IP that will bite you in the ### if given half a chance?

MxAngel




msg:4334026
 5:22 am on Jul 2, 2011 (gmt 0)

Couple of other IP's in the same block, heaps of different UA on 74.81.199.65

[botsvsbrowsers.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved