Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Freaky spiders



11:08 pm on Mar 30, 2001 (gmt 0)

This one is home grown.

UA: WebFountain
Host/IP: wfp2.almaden.ibm.com (
Pulled 112 pages in a reasonable length of time. Seems to obey the RES.

This one has perked my curiosity.

UA: unlostBot unlostBot@unlost.com.

Anybody ever use the Webclipping robot? I know what it is for but wondered if anybody used their service or had info on them. I've been getting hit from them every day for months.



9:32 pm on Apr 3, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

That unlost.com front page does grab your attention.


9:55 pm on Apr 3, 2001 (gmt 0)

10+ Year Member

I had been hit by webclipping.com as well for weeks (every day, deep spidering)

I wrote them numerous emails to ask if they could spider once a week and if they obey robots.txt.

No response. I ended just banning them in my .htaccess.


Featured Threads

Hot Threads This Week

Hot Threads This Month