Forum Moderators: DixonJones

Message Too Old, No Replies

Haste/0.11

Only visits once, but rips thru...

         

pendanticist

11:54 pm on Oct 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



216.127.84.21 - - [26/Oct/2003:10:45:40 -0800] "GET /robots.txt HTTP/1.1" 206 1524 "-" "Haste/0.11 (HOME: http*//haste.xatka.net/)"
216.127.84.21 - - [26/Oct/2003:10:45:40 -0800] "GET / HTTP/1.1" 200 20538 "-" "Haste/0.11 (HOME: http*//haste.xatka.net/)"

Their website makes a statement to the effect that it crawls a site only once and that it ...does not access any specific Web site at a high rate or volume.

I would hardly characterize the taking of 4-6 files per second as being slow.

Given that 'Haste' purports to visit only once, I'd suggest banning this puppy now before it rips thru your work.

Oh, and if you're lucky enough to have a ban in place against 'Haste', you'll be comforted with the knowledge that they say they obey robots.txt.

Now, isn't that special? A tad late to have much merit, but special none-the-less. Don't you think?

:-\

Pendanticist.

amznVibe

1:52 am on Oct 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



So Haste does make waste eh? Lol. banned - thanks for the alert!