Their results IMO are now the most relevant in the business. Now if they could tone down the sponsor's results just a bit they would put G out of IPO fervor.
AV has been on the move!
I'm getting referrals from them as well.
altavista is not only back in my logfiles -
their crawlers actually consume more than 85% of my
traffic - grabbing the same (huge) multimedia files
over and over.
right now altavistas crawlers are consuming several hundred
gigabytes. is anyone seeing the same? although i like altavista - if they continue im afraid that i have to ban their robots - the traffic they cause doesnt hold against the few referrals from this search engine (although i cant complain about the rankings ...)
here are the crawler id's/UA's:
any ideas? (or promises from altavista? :)
We have a new robots.txt forum [webmasterworld.com] where you could ask about keeping Scooter out of your large multimedia files, but allow it to crawl your other pages.
It'll be interesting to see what the AV-bashers have to say if the recent changes at G drive significant search traffic over to AV... I'm kinda glad to see an old friend back, myself.
[edited by: jdMorgan at 9:09 pm (utc) on Dec. 9, 2003]
Yahoovista listings anyone? :)
You might want to check out Yahoo!, I think they are flashing AV listings now, been in and out today.
thank you for the info - im actually trying to figure out if this is a new trend or not ... i guess the big question is: will the referrals/quality of altavista improve - or should we better ban their crawlers right now.
is anyone seeing the same?
and - the user agent is NOT scooter - the machines are listed above - the new user agent is "3.3.vscooter".
the crawler/grabber is "only" interested in multimedia files - if you have large image collections or huge video or audiofiles - beware!
actually there were other threads indicating bad behavior by the very same agent at:
another resource includes:
eventually we really have to block the entire thing -
not a clever idea from altavista. i can not appriciate it if they hit the same huge files over and over (most likely the get confused with all the mirror domains we own...)
|More Traffic Please|
As long as we are talking about AV, I used to be in their index for a long period of time. Now, it appears that I am not at all. Are they still picking up sites from their free submit option, or is the only real way to get in via express submit?
another piece of information that you might find useful:
action: i have deactivated/rerouted most of my mirror domains and i have banned altavista from my multimedia folder via robots.txt.
result: the bandwith consumed from altavistas multimedia
crawlers has been greatly reduced (from several gigabytes
to a few hundred megabytes)
conclusion: 1 - altavistas crawler are NOT capable to identify multiple instances of the same file BEFORE they download the entire thing (not sure if they do it afterwards either ...) - this could be a loophole to get multimedia content into altavista
2 altavistas crawler do not follow the robots.txt protocol - or it takes some time before they reread the file - another flaw in altavista's technology
3 i wrote to them two weeks ago (corporate marketing) and i am still awaiting an answer - their customer relationship management is not very responsive ...
In November a whopping 0.55% of my referrals came from AV. So far in December they are down to a more sustainable 0.4%.
Month after month Google plus Ink (plus, obviously, partners of both) give me around 95% of my SE traffic.