Forum Moderators: DixonJones

Message Too Old, No Replies

Anyone know what site grabber can do this?

Someone from Brazil used some weird site grabber on my site which....

         

kenpachi

6:57 pm on May 1, 2008 (gmt 0)

10+ Year Member



Someone from Brazil used some weird site grabber on my site which resulted in about 144MB of traffic in a short period of time. I noticed that the requests were seconds apart and the user-agent used would constantly switch between two user-agent strings, like it was trying to cloak itself.

Here's a chunk of user agents used:
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322)
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US;+rv:1.8.0.7)+Gecko/20060909+F
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US;+rv:1.8.0.7)+Gecko/20060909+F
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1)
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322)
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+I
Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US;+rv:1.8.0.7)+Gecko/20060909+F
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+I
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322)

Here's the grabbers IP:
201.86.251.172

Would you guys recommend that I block that specific address, or maybe a range of address? Is there a better to go about it, thanks.

JAB Creations

3:28 am on May 6, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Terminology is very important so please keep in mind cloaking refers to serving data based on the user agent.

What you intended to say was that the bot was failing miserably at spoofing their user agent.

I'd block it just for making a mess of my access logs to be honest. Be it an email scrapper, site scrapper, form spammer, etc if it's not a legitimate visitor (useful commercial bots and human beings) then you should block them.

- John