Forum Moderators: bakedjake

Message Too Old, No Replies

getting concerned about alexa

ia_archiever

         

seofreak

11:19 am on Jun 9, 2003 (gmt 0)

10+ Year Member



I mentioned in a post 2 days ago that alexa is crawling me like crazy. Till that day the bandwidth usage of my site was about 26MB for june. Today it has shot upto 1054MB straight! My monthly statistics have neer gone beyond 150MB. My total website size would be around 15MB .. Can anyone tell if alexa has caused this?

lazerzubb

11:21 am on Jun 9, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



<sidetopic>
what on earth does this have to do with Google news?
</sidetopic>

seofreak

11:25 am on Jun 9, 2003 (gmt 0)

10+ Year Member



Oh and My logs show hits for my internal pages (about 170 - size 80kb each) :

Host: 64.68.82.** : Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)

intikomi slurp, ia_archiever and 2 or 3 other bots

Is this the new IP of deepbot and could these be the problem?

Netizen

11:25 am on Jun 9, 2003 (gmt 0)

10+ Year Member



Look in your access logs, and if you want to stop it add the following lines to your robots.txt file

User-agent: ia_archiver
Disallow: /

See here [webmasterworld.com] for a discussion.

seofreak

11:26 am on Jun 9, 2003 (gmt 0)

10+ Year Member



I forgot to add the googlebot part .. dunno if this belongs here, sorry.

Netizen

11:27 am on Jun 9, 2003 (gmt 0)

10+ Year Member



64.68.82.* is the spider formerly known as FreshBot :-)