homepage Welcome to WebmasterWorld Guest from 54.196.24.103
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
businessdbbot
Turkey based business search engine
keyplyr




msg:4491000
 6:05 pm on Sep 3, 2012 (gmt 0)

UA: businessdbbot/v1.2 (http://www.businessdb.com/bot.php)
robots.txt: yes

BERIL TECHNOLOGY
89.107.225.0 - 89.107.225.255

 

dstiles




msg:4491035
 8:49 pm on Sep 3, 2012 (gmt 0)

I blocked the full beril range: 89.107.224.0 - 89.107.231.255

keyplyr




msg:4491051
 9:27 pm on Sep 3, 2012 (gmt 0)


What events led to to block?

wilderness




msg:4491068
 10:36 pm on Sep 3, 2012 (gmt 0)

You mean the criteria that it comes from the Class A is not enough reason ;)

keyplyr




msg:4491089
 12:44 am on Sep 4, 2012 (gmt 0)



I mean there are dozens & dozens of sub-range assignments within your ban. What specifically is the reason you banned them all?

dstiles




msg:4491356
 8:02 pm on Sep 4, 2012 (gmt 0)

Apart from them being a server farm hosting hundreds of domains?

Actually I'm not sure if they actually host the domains or just, in some way, route them - the whole range is given as RouterGate.

They've been blocked since before 2010 (when I created my current ban system).

keyplyr




msg:4491452
 12:58 am on Sep 5, 2012 (gmt 0)


As long as there are no small ISPs hidden in there. I've made that mistake numerous times. Sometimes the ISPs are added later, after I've done my research.

dstiles




msg:4491685
 7:39 pm on Sep 5, 2012 (gmt 0)

If they are that small - typically 512 IPs - is that such a problem?

blend27




msg:4494960
 1:40 am on Sep 14, 2012 (gmt 0)

businessdb

gets a 403 for a few month here now, I am not hiring..

But what does the site do? seems like another "" back-link scam...

lucy24




msg:4494966
 2:23 am on Sep 14, 2012 (gmt 0)

You mean the criteria that it comes from the Class A is not enough reason

89 = the entire continent of Europe plus the bits of Asia that are covered by RIPE
89.107 = ditto, broken into /21 chunks, still covering the map
89.107.224-231 = Turkey, apparently further subdivided into /24 segments that make me regret I don't read Turkish, though I'm sure someone hereabouts does.

?

I haven't been much plagued with Turkish robots. Even get the occasional legitimate visitor. Well, possibly a mistaken visitor looking for something else, but that's not their fault. Turkey may not be amenable to wilderness's particular widgets. I would have to look it up :(

revrob




msg:4499600
 7:13 am on Sep 25, 2012 (gmt 0)

You may want to pay a bit more attention to Turkey as controversial company PHORM are now operating their Deep Packet Inspection/Content scraping and communications intercepting Targeted ad business there with Turkish ISP monopoly TTNET. [enphormasyon.org...] has information about the campaign against them.

(The EU have declared their consent model non-compliant with EU law and they had to leave UK after negative publicity in 2008)

I had BERIL TECHNOLOGY on my site this morning looking like a human visit except for a final wget request with a totally different useragent which triggered my WGET bans and made me check it out.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved