Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & keyplyr

Message Too Old, No Replies


4:14 am on Mar 3, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
votes: 890

UA: IABot: Checking if link from Wikipedia is broken and needs removal - See https://meta.wikimedia.org/wiki/InternetArchiveBot/FAQ_for_sysadmins
Protocol: HTTP/1.1
Robots.txt: No
Host: Wikimedia Foundation Inc. -

This same bot now scrapes for the Internet Archive (IA.) Some site owners feel having their digital property (their entire web site) taken, stripped of ads & affiliate links, put on the IA and made public violates their rights and infringes copyright.

Clever, it appears they now require IA access in order to retain your Wikipedia citation and back-link.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members