bitmagicbot

[Pfui]

No G results at all for this robotic critter:

ec2-184-73-234-197.compute-1.amazonaws.com
bitmagicbot/0.1 [e-mail]

robots.txt? NO

[incrediBILL]

That's because, best I can tell, it's brand spanking new!

Just hit my server for the first time today.

[Pfui]

My, my. Aren't we the lucky ones?! (pfffffft:)

[keyplyr]

The amazonaws cloud has a silver lining in that it seems to attract a large bunch of bad bots all in one banned basket :)

[Pfui]

FYI, about an apparent connection between this bot and atypical Apache log entries. As first noted here:

amazonaws.com plays host to wide variety of bad bots
[webmasterworld.com...]

>>
...[Thus far this month, the] ONLY hits from the exact same AmazonAWS address --

ec2-184-73-234-197.compute-1.amazonaws.com

-- are ALL malformed, UA-wise... until one brand-new bot suddenly makes an appearance in the "UA" spot:

[

03/Aug/2010:12:58:31 -0700] "GET / HTTP/1.1" 403 1468 "-" ""
[04/Aug/2010:04:10:08 -0700] "GET / HTTP/1.1" 403 1468 "-" ""
[04/Aug/2010:06:46:52 -0700] "GET / HTTP/1.1" 403 1468 "-" ""
[04/Aug/2010:06:51:37 -0700] "GET /dir HTTP/1.1" 403 1468 "-" ""
[04/Aug/2010:10:24:00 -0700] "GET /fileA.html HTTP/1.1" 403 1468 "-" ""
[04/Aug/2010:12:12:16 -0700] "GET /fileB.html HTTP/1.1" 403 1468 "-" ""
[05/Aug/2010:12:18:08 -0700] "GET /FileB.html HTTP/1.1" 403 1468 "-" "bitmagicbot/0.1 [e-mail]"

<<

If you've been visited by 184.73.234.197 this month, and/or bitmagicbot, could you kindly eyeball your raw logs and see if any of the hits show similar (or odder) missing/altered 'UA position' oddities? TIA

[incrediBILL]

I don't think you can make any sense out of AWS logs like this because they're cloud computing so it's anyone's guess when it comes to anything without a label.

[Pfui]

Bill, we have differing views re AWS. Cool. So let's say bitmagicbot came a-callin' from any server farm's IP:

When the exact same IP newly appears and hits with the exact same never-seen-before string x2 days, then suddenly on day 3, behold, a brand-new bot -- coincidence? Hm.

So hey, I'm just asking about the baby bot's hits, Bill. For example, it hit you (from AWS?):

Did you have prior visits from the bot's IP this month, before it appeared by name? If yes, what was the UA string, if any, please?

[incrediBILL]

We don't have differing views about AWS at all, it's a cesspool.

I get lots of overlapping hits from AWS for different UAs, same IP, which is why I'm not willing to throw bitmagicbot under the bus for that, but I'll make an exception for their lack of robots.txt request.

It's possible they forgot to set the UA at first, but like I said, AWS is such a cesspool it's really hard to point the finger at just one bot and blame it for something it may or may not have done.

However, in this case you may be right, anything is possible.

[tangor]

There are times we "shoot first" and ask questions later. Providing an oddity: asked for bots, took it, and left me alone.

YMMV

[Pfui]

On another site during the night, its first magical hit. Same IP in OP, still no robots.txt --

ec2-184-73-234-197.compute-1.amazonaws.com
bitmagicbot/0.1 [e-mail]

robots.txt? NO