Forum Moderators: open

Message Too Old, No Replies

AAfter.com Crawler

         

keyplyr

8:47 am on Sep 18, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: AAfter.com Crawler/AAfter-1.0
robots.txt: yes
rDNS: adsl-99-3-77-**.dsl.rcsntx.sbcglobal.net
[Verified]

This bot is very focused, well-behaved, and wants to do good to internet community. For any questions, please call collect [number omitted]. Team AAfter, Dallas, TX, USA; http://aafter.com/crawler.htm; crawler at aafter.com

(omission mine)

Bot info page is non-existent, forwarding to their home page. They also offer an "Anonymous Proxy" tool to "do good to internet community."

subhankar ray

10:00 pm on Sep 28, 2009 (gmt 0)

10+ Year Member



Keyplyr,

Thank you for noticing AAfter Search.

"Anonymous Proxy" link is an external link, and no different than Google/Yahoo providing links to different proxy sites. However, we have a ton of other useful features [ see 'how to use' link] how we are helping internet community.

Cheers!

Subhankar Ray

keyplyr

9:50 am on Sep 29, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Subhankar Ray, thanks for posting.

Well IMO there is a significant difference between the usage of your referral credit link to the "Anonymous Proxy" tool and Google/Yahoo's search result pages that may list proxy services for relevant search terms.

Anonymous Proxies, referral credited or otherwise, are not considered helpful to many webmasters for obvious reasons. This immediately caused a negative reaction when 1st, I was redirected from the false URL in the UA string, then I see the "Anonymous Proxy" tool link.

Consider that webmasters have become weary of non-existent and misleading URLs in user agent strings. You should consider correcting that if you want to be seen as as friendly or "helpful." What we like to see is a correct path to an info page that details exactly what you do with the data you harvest and how it benefits webmasters who decide to allow your bot access to their server's resources.

Until then, I'll continue to block requests from this range and this UA.

subhankar ray

11:31 pm on Oct 23, 2009 (gmt 0)

10+ Year Member



Keyplyr,

Just to let you know we have fixed the link so that you can get information about the AAfter search crawler. Let us know if that is enough. Considering our UA string has a phone number, and we have not received any 'angry' call yet, it must be behaving as planned.

We are still debating about the proxy link. It is also used by webmasters managing PPC campaigns to check different ads. Hence, we are making some of them happy.

Regards,

Subhankar Ray