Welcome to WebmasterWorld Guest from 54.159.179.132

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Freaky spiders

   
11:08 pm on Mar 30, 2001 (gmt 0)



This one is home grown.

UA: WebFountain
Host/IP: wfp2.almaden.ibm.com (198.4.83.49)
[swcp.com...]
Pulled 112 pages in a reasonable length of time. Seems to obey the RES.

This one has perked my curiosity.

IP: 212.37.219.147
UA: unlostBot unlostBot@unlost.com.
[unlost.com...]

Anybody ever use the Webclipping robot? I know what it is for but wondered if anybody used their service or had info on them. I've been getting hit from them every day for months.

[webclipping.com...]

9:32 pm on Apr 3, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



That unlost.com front page does grab your attention.
9:55 pm on Apr 3, 2001 (gmt 0)

10+ Year Member



I had been hit by webclipping.com as well for weeks (every day, deep spidering)

I wrote them numerous emails to ask if they could spider once a week and if they obey robots.txt.

No response. I ended just banning them in my .htaccess.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month