Alexa Problem

Not in Ranking but in the Robot

[sidyadav]

In my site Error logs , I saw:

[Sat Sep 13 06:25:23 2003] [error] [client 209.237.238.159] File does not exist: /something.html
[Sat Sep 13 06:25:23 2003] [error] [client 209.237.238.159] File does not exist: /something2.html

So they are requests from 209.237.238.159 which resolves to crawl9-public.alexa.com and is the Alexa robot.

But, in my logs , there is not sign of ia_archiver or any other Alexa robot , and there is not visitor from the IP: 209.237.238.159.

I don't know if its a problem of my logs or is it ia_archiver not leaving its information behind.

(ia_archiver is banned from my site)

Please help me out.

Sid

[jeremy goodrich]

Sid - that *is* alexa.

Either they found a link to those files on your site - or - they have some problems with their crawl routines.

And, even if you ban ia_archiver from your site / if you don't do it via .htaccess with a rewrite rule of some kind they'll still crawl your site, afaik, as they don't obey robots.txt.

[sidyadav]

I didn't ban them from robots.txt , but I just banned them from .htaccess file and I have a IP blocker in my control panel , I used that and banned all the IP's of ia_archiver I know of (except for this one) Now, I've banned this one, Hope it works well.

Sid