homepage Welcome to WebmasterWorld Guest from 54.204.67.26
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Mbbf 0.1 Dwl
Wikimedia
keyplyr




msg:4478495
 6:16 pm on Jul 24, 2012 (gmt 0)



UA: MBBF 0.1 DWL
robots.txt: no


Wikimedia Foundation Inc.
91.198.174.0 - 91.198.174.255
deny from 91.198.174.0/24

 

dstiles




msg:4478518
 8:24 pm on Jul 24, 2012 (gmt 0)

I've blocked the whole 91.198.0.0/15 - same "company" but many countries.

First complete blocked was July 2010 with the note:

"Various countries/nets - Wikimedia Foundation Inc and others"

keyplyr




msg:4478560
 10:35 pm on Jul 24, 2012 (gmt 0)

I'm wondering if these hits may also include link checking. I have a dozen or so incoming links from Wikipedia and don't want to loose the traffic.

dstiles




msg:4478887
 9:56 pm on Jul 25, 2012 (gmt 0)

Would you lose traffic by rejecting their bots? As far as I know traffic comes in on its own IP/UA - but I could be wrong. :)

Whatever, I have those IPs blocked and last year one site received a couple of thousand hits from five or six wikis (different countries) (last year was latest stats processed for this site).

keyplyr




msg:4478897
 10:51 pm on Jul 25, 2012 (gmt 0)


Would you lose traffic by rejecting their bots? As far as I know traffic comes in on its own IP/UA


My question was...

I'm wondering if these hits may also include link checking.


Didn't want my incoming links from Wiki* to be invalidated and removed. Once removed it takes a human editor to put it back in. I've had numerous issues with them for years. Every time I find my plagiarized content I remove it, only to see it again a week or two later. Every time I (as an editor) insert my link as a source (since they continue to use my content) the link gets removed by another editor.

It's as bad as the Open Directory. In some areas/topics, it's dominated by only several editors who give outgoing links to favored sites and block any other editors from adding/changing things.

I've pretty much let go of that game but don't want to loose my dozen or so links that remain, thus my concern about this bot being used for link validation.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved