Welcome to WebmasterWorld Guest from 54.147.44.13

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Punk Spider

caught this punk snooping from the clouds

     
1:22 am on Jul 8, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14622
votes: 85


184.106.80.222

"Punk Spider/PunkSPIDER-v0.1"

robots.txt: YES
2:27 am on July 8, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


"Punk Spider"

Arent they all ;)
3:01 am on July 8, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12696
votes: 244


v0.1

Oi! Save your alpha testing for your friends' sites! Come back when you're ready with v. ... well, OK, at least 0.8.
6:29 pm on July 8, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3091
votes: 2


Server range (Rackspace). Already blocked.
6:42 pm on July 8, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14622
votes: 85


Server range (Rackspace). Already blocked.


That too.

I attempt to be polite with robots.txt set aside as a special case in the firewall so the dynamic robots.txt code will serve up permissions to anyone that asks. However, if you ask for robots.txt and are denied, and then request any other page from that user agent or IP address, you're also denied since the script enforces the robots.txt rules.

The reason I track the IP is some smart ass started asking for robots.txt using one user agent to test the waters then switched the user agent when asking for pages so I started tracking the IPs making the robots.txt requests if they're denied :)

The data center blocking, which includes rackspace, applies to all other files :)

It's complicated yet so simple.
9:21 pm on July 9, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3091
votes: 2


Your system is far more sophisticated than mine. :)

My IIS system (sans htaccess) can only intercept IPs and headers on ASP page access. Far too late for me to change it now. :(
7:23 am on July 21, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14622
votes: 85


That PUNK came from a new IP today: 198.101.170.228

Wondering if they're in the cloud.
10:14 pm on July 21, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3091
votes: 2


Don't care. Rackspace: blocked 198.101.128/17 :)

Does rackspace operate a cloud? If so, where? What IPs?