Forum Moderators: open

Message Too Old, No Replies

Mp3Bot

         

keyplyr

9:03 am on Nov 3, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: Mozilla/5.0 (compatible; Mp3Bot/0.6; +http://mp3realm.org/mp3bot/)
robots.txt: no
mp3realm.org WHOIS shows Cayman Islands.

Coming from 2 IPs at once:

ost2.marketzenith.com
HostRocket Web Services
66.147.224.0 - 66.147.239.255

ns176.midphase.com
Hosting Services, Inc
206.217.192.0 - 206.217.207.255

Last time this bot came 'round, I was very busy and as I have a music themed site, I neglected to investigate. Can anyone verify that this bot from these sources is what they say it is?

Pfui

5:48 pm on Nov 3, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Misc. bits from my notes (non-music sites)... The following UA crawled from mp3realm.org:

Mp3Bot/0.2 (http://mp3realm.org/mp3bot/)

The following UAs crawled from all over, almost all server farms including ad nauseam hits from HostRocket-based Hosts, its own (hrwebservices.net; hrnoc.net) and numerous others, so many that sometimes the UA resembles an exploit:

user_agent=Mp3Bot/0.1 (http://mp3realm.org/mp3bot/)
Mp3Bot/0.1 (http://mp3realm.org/mp3bot/)
Mozilla/5.0 (compatible; Mp3Bot/0.4; +http://mp3realm.org/mp3bot/)
Mozilla/5.0 (compatible; Mp3Bot/0.6; +http://mp3realm.org/mp3bot/)

UAs often used simultaneously, as opposed to upgraded over time. I've got it 403'd all over but for robots.txt which it asks for -- sometimes.

keyplyr

11:14 pm on Nov 3, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks Pfui

MP3realm.org:

Mp3Bot is a distributed crawler meaning many hosts on different addresses run the indexer. Also due to DHCP, dynamic IP's are constantly renewed and refreshed with a new address periodically. You are better off using the Robots Exclusion Standard to block the bot.

Except out of the 56 requests to the server, none were for robots.txt from either of the two IP ranges using MP3bot in their UA string. Regardless, if it's a distro then there's too much room for abuse, except in a select few.