Welcome to WebmasterWorld Guest from 22.214.171.124
Is anyone familiar with this bot?Thanks vegasrick - yes, A6-Indexer is A6 Corp's bot. Don't know why you consider it "an attack." All they are doing is requesting files. That's what bots do.
[edited by: keyplyr at 5:18 am (utc) on Jul 12, 2018]
(honestly now, it’s not as if they can say “Oh, sorry, I didn’t know you meant me” when that is their complete, litteratim UA string) it seems a bit disingenous for them to claim they will go away if you ask them to. Isn’t that like spam email that professes to honor an opt-out when you never opted-in?
millions of threads at rapid speedThat's a pretty good trick considering AWS has limiters for abuse of this nature. This must be something unique and I would be persistent and prove to AWS exactly what happened.
A6 Corporation maintains the Internet Topology, a living map of the internet that tracks 13.5 billion relationships between URLs, calculating the rank of each web page within one or more subject realms. The Internet Topology is used by online advertisers to increase response rates and avoid fraud, and by website publishers to increase revenues on...source: crunchbase.com/organization/a6-corporation
Recommend a permanent deepsixWhy? Just because you see no benefit from this bot you are recommending that everyone else also block it?
Assassination prior to investigation?More like termination with extreme prejudice. They caused a problem on my site trying to download thousands of pages. They used multiple IP addresses to evade limiting. They were deepsixed. And I don't believe their waffle about Internet Topology either.
This does not look like a legitimate operation with the best interests of the webmasters at heartThat's an unreasonable statement. Why would any botrunner have "the best interests of the webmasters at heart"?
[edited by: keyplyr at 10:22 pm (utc) on Jul 22, 2018]
[edit reason] removed derogatory statement [/edit]
Do the research
Most bots do not support robots.txt. That is not applicable to whether they are beneficial to the site owners interests. Unfortunately, some beneficial agents do not include a comprehensive mission statement. That's why it's necessary to do the research.To point it out again. Their website is a splog. There's no contact details or "about page" in the UA. Their Twitter a/c is dead since 2014. Their Facebook page has been deleted and ditto for their Linkedin page. All that's left is some rubbish about Internet Topology in a Crunchbase article from 2011. To use an old New York expression: A perp is a perp is a perp. :)