homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Very agressive robot?
Got hammered by this nasty little bugger... check it out.

 12:59 pm on Apr 27, 2001 (gmt 0)

Don't know who this is but it traced back to an ISP in the UK called Planet Online Unlimited. I wrote to them about it and hopefully will get some reply. Here is the information on the bot.. - - [27/Apr/2001:03:04:47 -0600] "GET /fortuna.htm HTTP/1.1" 200 9181 "http://www.google.com/search?q=Sors+salutis+et+virtutis&btnG=Google+Search" "Mozilla/5.0 (X11; U; Linux 2.2.16-22smp i686; en-US; 0.7) Gecko/20010129" - - [27/Apr/2001:03:05:30 -0600] "GET /Moon-ani.gif HTTP/1.1" 200 93875 "" "Mozilla/5.0 (X11; U; Linux 2.2.16-22smp i686; en-US; 0.7) Gecko/20010129"

Then in proceeded to request this file and one other over and over again about once every 60 seconds or so for several hours. The other file was also a GIF and it did not check robots.txt.





 2:02 pm on Apr 27, 2001 (gmt 0)

It looks like someone may be using the Gecko engine to create a bot, for what purpose i do not know. I found some browser stats going back to Monday 05/Feb/2001 on a Google search for Gecko/20010129 so it's been around for at least a few months, but if this guy hit you that many times I would say they have a bit of fine tuning to do on their project. If writing them yields some answers, let us know!

descr: Planet Online Limited
descr: The White House
descr: Melbourne St.
descr: Leeds LS2 7PS United Kingdom

rev-srv: earth.theplanet.net
rev-srv: venus.theplanet.net
rev-srv: pluto.theplanet.net

See [theplanet.net...] which is an e-business solutions company.

(From Netscape )

"...Gecko is Netscape's revolutionary next generation browser engine. It features industry-leading, fully compliant standards support (including HTML 4.0, XML, CSS, and DOM) and is small, fast, and modular."

a search on Gecko/20010129 also gave these results.
(Galeon is a Gnome based browser)...
"Galeon gives the following UA string: Mozilla/5.0 (X11; U; Linux 2.4.1 i686, en-US; Galeon) Gecko/20010129"


 2:24 pm on Apr 27, 2001 (gmt 0)

This morning I had a very similar situation. Someone in France using Netscape 6.01 (uses Gecko 20010131) on NT was requesting a single gif file over and over, with the rate increasing to 20-30 requests a second by the time I blocked the IP.

I suspect, given SugarBane's experience, that there's a bug in Gecko somewhere and these are ordinary surfers rather than bots.


 2:48 pm on Apr 27, 2001 (gmt 0)

Hmmm, I hope that's all it is. From Melbourne to France, and the common thread is Gecko and extreme activity. My increasingly suspicious and sometimes paranoid mind is thinking that perhaps some cranksters are using the gecko engine to perform some sort of low level denial of service attack???



 3:05 pm on Apr 27, 2001 (gmt 0)

>low level denial of service

LOL - it wasn't so low level around here, my service was entirely denied for a couple of minutes ;)

The thought of a deliberate attack did enter my mind (I must be just as paranoid), but it seems an elaborate way to do it.

I'll still be keeping a close eye on my bandwidth stats over the next few days though....


 5:59 pm on Apr 27, 2001 (gmt 0)

SugarBane, it looks like it was in fact a linux box: * Linux 2.0.35 to 2.0.9999 :)
On purely a gut call, I bet it is a Mozilla bug.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved