Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

Wotbox/2.0 is cheating from DSL hosting

Contains unauthorized content

4:20 pm on May 6, 2013 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
votes: 99

Wotbox/2.0 (bot@wotbox.com; http://www.wotbox.com)

Wotbox/2.0 (bot@wotbox.com; http://www.wotbox.com) (AYIMA)

Robots.txt: YES

While it asks for robots.txt under this user agent it is obviously using other user agents to gain access. Since I whitelist this UA is never getting access yet some of my pages, from an older site using older code, still ended up in their listings but it had clues.

First, the page is tagged with a code that it passed the UA test meaning they were masking themselves as a browser to collect the data from the following IPs:

inetnum: -
netname: Bulldog
descr: 40:1 Dynamic IP Pool
country: GB
role: Cable and Wireless Access Ltd
address: SE1 0SL

Sorry I can't provide more details but it if passes all the other filters the most I do is identify the source of the crawl and not the user agent. Working on something better but I didn't want to maintain that amount of forensic data on my end as it gets a little crazy after a while, esp. since you don't often find the destination of where the data ended for months which makes for a LOT of storage.

Maybe I should try a new rule so that anything in the reverse DNS with "host" or "server" gets the axe and see just how many cyber ships get stranded on that reef as it would certainly help locate hosting within normal service provider ranges which is always problematic at best.
8:18 pm on May 6, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
votes: 8

Thanks for the info. I've had problems from bulldog IPs in the past, though not recently.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members