Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

Freaky spiders

11:08 pm on Mar 30, 2001 (gmt 0)

Full Member

joined:Jan 24, 2001
votes: 0

This one is home grown.

UA: WebFountain
Host/IP: wfp2.almaden.ibm.com (
Pulled 112 pages in a reasonable length of time. Seems to obey the RES.

This one has perked my curiosity.

UA: unlostBot unlostBot@unlost.com.

Anybody ever use the Webclipping robot? I know what it is for but wondered if anybody used their service or had info on them. I've been getting hit from them every day for months.


9:32 pm on Apr 3, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 17, 2000
votes: 0

That unlost.com front page does grab your attention.
9:55 pm on Apr 3, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 3, 2001
votes: 0

I had been hit by webclipping.com as well for weeks (every day, deep spidering)

I wrote them numerous emails to ask if they could spider once a week and if they obey robots.txt.

No response. I ended just banning them in my .htaccess.


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members