Forum Moderators: open

Message Too Old, No Replies

about.com

         

keyplyr

11:30 am on Nov 15, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Stealth crawling (or link checking?)

rDNS: mxc1s.about.com
UA: Mozilla/4.0
robots.txt: no

207.241.144.0 - 207.241.159.255
207.241.144.0/20

dstiles

9:56 pm on Nov 15, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



They've been blocked for a long time here. I also tried to remove entries they'd crawled before I caught them - devil's own job but they removed them if I insisted enough.

keyplyr

1:56 am on Nov 16, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



They've published a couple articles of mine along with my embedded links (he he) so I don't block them - just posting about the "stealth" stuff.

dstiles

8:50 pm on Nov 16, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I never liked the way they scraped content from a web site and republished it as if it were a genuine submission by the site owner. And then got far too many entries in the SERPS for MY sites! :(