homepage Welcome to WebmasterWorld Guest from 54.226.173.169
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Pagepeeker.com
bad bot
cyberdyne




msg:4412265
 10:51 am on Jan 30, 2012 (gmt 0)

Pagepeeker .com - "Put website thumbnails and favicons on your site. Free branded (unlimited impressions), free unbranded, paid versions. Fast servers, lots of impressions included."

=

A few months ago I noticed a lot of unnecessary extra activity from this bot so added it to my robots.txt using: Disallow: /

Recently, it has been on a mission, hitting my page dozens of times in a 48 hours period despite still having access to robots.txt, thereby ignoring its' rules, so it is now blocked it via .htaccess

Anyone else seen over-enthusiastic activity from this bot ?

 

Pfui




msg:4412374
 5:17 pm on Jan 30, 2012 (gmt 0)

Just always-bad activity, from it and its notoriously bad ISP. [webmasterworld.com...]

cyberdyne




msg:4412380
 5:24 pm on Jan 30, 2012 (gmt 0)

Thank you for confirming.

keyplyr




msg:4412453
 8:49 pm on Jan 30, 2012 (gmt 0)


hetzner/your-server, Africa
188.40.0.0 - 188.40.0.31
188.40.0.0/16

cyberdyne




msg:4412592
 8:18 am on Jan 31, 2012 (gmt 0)

Also
46.4.120.2
46.4.32.18

keyplyr




msg:4412633
 11:10 am on Jan 31, 2012 (gmt 0)

deny from 46.4.0.0/16 should take care of all these hetzner/your-server ranges:

46.4.0.0 - 46.4.0.31
46.4.32.0 - 46.4.32.63
46.4.120.0 - 46.4.120.31

cyberdyne




msg:4412635
 11:15 am on Jan 31, 2012 (gmt 0)

Thanks keyplyr, although I will say one thing in the bots defence; at least it always identifies itself as User-Agent: PagePeeker(*) so that will also work as an effective block ...for now.

keyplyr




msg:4412637
 11:26 am on Jan 31, 2012 (gmt 0)


@cyberdyne

IMO a lot of pests come from hetzner/your-server, in all their ranges/locations.

Here are some more:

78.46.0.0 - 78.46.255.255
78.46.0.0/15

85.10.192.0 - 85.10.207.255
85.10.192.0/18

88.198.0.0 - 88.198.15.255
88.198.0.0/16

176.9.0.0 - 176.9.0.31
176.9.0.0/16

178.63.0.0 - 178.63.0.63
178.63.0.0/16

188.40.0.0 - 188.40.0.31
188.40.0.0/16

213.133.96.0 - 213.133.111.255
213.133.96.0/19

213.239.192.0 - 213.239.199.255
213.239.192.0/18

cyberdyne




msg:4412640
 11:41 am on Jan 31, 2012 (gmt 0)

Duly noted ;)
Thank you.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved