|Alexa Review Issues|
Failure to validate quality reviews
I could not find this posted anywhere else on Webmasterworld.com so hope I have posted this in the right area.
I have long been aware of Alexa have myself positioned quite well on the engine. However in the last months I have noticed reviews showing up for my business site. The first two impressed but who were these reviews from? Perhaps a customer who cloaked their name to keep their privacy.
Recently last week while reviewing my stats I noticed a very negative review posted about a month ago. Again from a person I have never heard of. I brought this to the attention of my partner who is a research queen? she was able to find a review on our competitor's site on the same date. The ironic thing about the post on our competitor's site is that it is exactly the opposite and related to the review for our site. Amazing! I will clear the air here in saying that I have had not one dissatisfied customer in my 5 years of business so that rules out the negative review being from a client.
The negative review is not what bothers me because it is an uneducated review. It is Alexa's poor review system that fails at making reviews creditable.
Alexa Reviews need to be Reviewed!
To see how this system works, I did the honorable thing by trying to submit my own review to webmasterworld.com's listing. It was easy and don't worry :) it was positive. What is scary is I could have said anything and used any e-mail address I wanted as long as I was there to confirm the review. Meaning anyone with a Hotmail or Yahoo e-mail address can submit a review.
What is the point of Alexa review when it is not actually backed up from a creditable or qualified source? What would stop me from doing the unethical and posting negative reviews against my competitors? I am a strong believer in Karma so I don?t go there, but others do so it seems.
I contacted Alexa regarding my concerns and I got a reply stating that all reviews are reviewed and approved before posting them. Even after showing them my example of the 2 similar posts that would without reason clearly show it was my competitor reviewing against me. I communicated clearly that I don?t want either of the reviews good or bad if I don?t know they are from a creditable source.
How can the quality of reviews be improved? I don?t want them if the reviewer did not go through more qualifications to post the review. Further, I think Alexa is going to model them selves after Google they want quality results correct? What is the point of reviews anyway?
>>> except to the majority of their users. Who happen to live in Korea.
That's interesting. Are you sure about that?
Wired2mac: I wouldn't lose sleep over this.
If a large number of your prospects are Alexa users, you should see some original ratings posted by your customers. That would compensate for the odd 1 star ratings given by your competitors and other oddballs.
Alexa reviews are like Download.com's software reviews. Half of the reviews are posted by the software publisher and the other half by their competitors. I guess if you can't beat 'em, join 'em.
>>some original ratings posted by your customers
And some original rantings posted by your competitors.
I ban Alexa's bot and ignore their stats for any practical purposes.
Thanks for your replies... I feel it is Alexa's obligation to allow site owners an opt out for reviews. I don't want them good or bad unless they are from someone reliable.
Lack of thought for their review program only confirms what has been said thus far about Alexa. However many professionals do take Alexa results seriously. Since I do get some traffic from their site, I have to act from a professional stand.
Alexa may just be not as smart as they should be, because not much development thought went into their review system.
I've used Alexa for nearly 2 years and I think it's great.
I don't really understand why it's called spyware. Anyone using it knows its purpose, and has no problem with being tracked, and the sites which it crawls are crawled by zillions of other bots too, including google.
Who are they spying on?
We use Alexa as a marketing tool. It is one of the few ways to track our progress aginst that of your competitors.
That said, I hate the reviews. It is a sad fact of life that malicious posters use the comments to post negative reviews of sites. Many posts are made by disgruntled ex-emploees and competitors. This hurts the credibility of all reviews.
I would like site owners to have review of the comments, or better still, no comments at all.
Getting back to if Alexa has a valuable to a webmaster or not--SURE IT IS.
Yes, there are too many sites ranked at the top from the Korean market. However, I personally don't know anyone who is honestly fighting to be in the top 20 worldwide.
The value is that the order is the same and the playing field is equal for you and your direct competing sites. I use Alexa to see where I am in direct relation to my closest competition. I don't care who is 18th globally of all users--those sites aren't in my business trade.
The trends section and compare your site to another site are what I use primarily. I don't worry about how many users of each million online.
I look at me and those in my business and see who is climbing and who is falling.
For that valuable task, Alexa is golden.
About the reviews--yes the system is about as imperfect as the Florida election vote in 2000.
Personally I feel that Alexa offers a good service here and a negative review isn't always bad.
Just check the ups & downs for WebmasterWorld and you'll see want I mean.
If one bad review is the only thing there -- well that can be a problem - but 1 or 2 bad ones and 20 good to exceptional ones indicates what?
A competitor that needs to the do this obviously has a problem with their own end thus needs to make it look like you are the problem. Generally this tactic really isn't effective - other than wasting the competitors time so they don't improve their own service - thus helping you in the long run.
Just the same an actual visitor providing "bad" feedback is good for your business - provides much didn't insight from an unbiased external source.
You could always request -- since Alexa captures the email of the poster - that they forward a message to the reviewer indicating your wish to improve their experience.
If they do and it was a competitor I doubt you will hear from them - but an actual website visitor/customer you probably would.
Any system that allows open reviews of this nature has the same problem. for ex. - amazon.com,citysearch.com,...
Good point, Tommy. Amazon has been dealing with the review problem for years, although perhaps authors see less value in trashing their competition than site owners. I think Amazon's problems are mostly bogus positive reviews posted by the author & friends. "Most informative book on this topic I've seen. Author is very knowledgeable and speaks with authority, but communicates the information in an entertaining and easy-to-understand manner. A must buy for anyone interested in this topic!"
I think zagat has the best solution to this problem - Have an editor pick and choose comments from a whole bunch of reviewers ( at least thats how i think it works). But that costs money and thats probably why their site is not free anymore.
Please see this thread and message #13 by Duncan:
|As of a couple weeks ago alexa made a change. Now, when you write a review, your email address has to be validated by alexa AND, alexa will delete any prior reviews from email address for that site. Ouch. |
That's the point. It's not accurate.
|one of the few ways to track our progress aginst that of your competitors. |
You feel it's accurate because you use it, and Alexa gives your rankings a boost because of it. By using the tool you are giving yourself a boost. You see yourself higher ranked and it gives you a warm glow. But it's a lie, and you're lying to yourself.
It gives you comfort to see your site performing better than your competition, but it's an illusion. If it's good news, then you are favorably disposed to it. This is human nature, to be susceptible to flattery and vanity. But that's all it is.
Similar to when Google ignores a web site, web site owners complain that it's algo is no good. If Google ranks them then the owners think the Google algo is incredibly fair.
To accurately, track your competitors all you have to do is create a spreadsheet with all your relevant keywords then list their positions month to month. Surely you're doing one for yourself? Just duplicate it for your competitors. That, is accurate tracking.
With much respect to your opinions, I neverteless have to respectfully disagreee and assert that using Alexa for the purpose of tracking competitors is lazy, inaccurate, and self-delusional.
it is all about creditable reviews
|As of a couple weeks ago alexa made a change. Now, when you write a review, your email address has to be validated by alexa AND, alexa will delete any prior reviews from email address for that site. Ouch. |
What I learned by posting a review about webmasterworld in Alexa, I was able to use a hotmail address. All I had to do was confirm that address and my post was approved. Spam - everyone who does it, including reviewers, has at least several yahoo, hotmail accounts on hand. How would one solve that issue?
Okay it is spyware ~ so if non of the issues can be resolved to make reviews legit maybe I can get removed from their database?
|Who really cares about Alexa? |
I never asked to be listed in Alexa, but I understand how my listing got there. Do I want to be removed from their site? (could i anyway?) I don't mind the listing, after all it is a link and even if my site visitors from Alexa are small, it is a customer and so to that ~ Alexa is important to me.
However, I don't want reviews on my site. Good or Bad, if they are not from a creditable source. I like the feedback here regarding this topic, and quite shocked at how many replied! Clearly this is an issue because people do care one way or another.
Alexa is not on my good side... But then who cares. ;)
(1) I also have experienced negative reviews on Alexa made by people who have obviously never used my site. Amazingly posted on the same day that my competition gets a glowing review.
(2) Seems to me that what happens is that someone posts a negative review, then the webmaster posts some nice reviews to balance it out (all on the same day). Check out www.vrbo.com on Alexa for a good example of that.
(3) End of the day the review stars are useless, but it's worth checking to make sure someone is not muddying the waters.
One of my sites is a pay-for-inclusion niche directory. I've had prospective customers tell me they don't want to list because my Alexa rating is lower than competitors. I install the toolbar, use it for a couple of weeks on my site, presto I have a better rating and I make a $300+ sale.
How many, though, potential customers use Alexa and don't let me know? How many of them take the reviews seriously?
Alexa data are crap and instead of making better disclaimers they do all sorts of things such as these new ranking banners to make their results appear more authoritative.
Shame on Alexa - Shame on Amazon
If you look at the Alexa comments for dmoz.org it's a little like a rerun of the Directories forum.. except there are a few good comments in there.
The problem is that the technology doesn't quite work. He's a practical example from a webmaster point of view.. scouting out link partners. (Warning: topic drift)
Let's say you have a site about garden gnomes and want to find other sites that would be worth getting listed on. One way is to use the ODP to find similar sites, but it's a bit hit and miss... so instead, you use the Google directory which lists the site by PageRank so you can scout out the best sites and ignore the useless ones.
What Alexa *should* allow you to do with the way it now uses ODP data is allow you to scout out the most *popular* sites in a category, but when you drill down into the directory it keeps thinking that crappy Geocities pages are the most popular because of the way the Alexa algo defines "site". In other words, it's pretty much useless.
Personally I think Alexa is quite fun, especially when comparing against competitors as it is really the ONLY way you can get a rough idea of other people's traffic levels. The reviews always *were* open to abuse and at least it's slightly harder now, but to be honest they're probably best ignored or even ditched.
Using Alexa for site tracking is neither lazy nor self-delusional.
Due to a change in our domain names over the past six months our Google ranking dropped from 13th for our main keyword to 36th, even though we combined our domains and added new features giving an extra 250,000 page views/month. We now have to wait months to see if our Google rank will recover. Our total page views are higher than the site that occupies the No 4 position, so your method does not work for us.
In the same time our Alexa rankings from 30,000 to around 6,000. Our competitors have stayed static because they have not changed their sites.
Our clients, staff and readers love the graphs Alexa makes. It makes sense to them becaue it is independent and can be more easily understood than the methods you mention. It is easier to understand than Google's system, which is slow-moving and has many problems if you change domain names as we have.
You may not like it, but Alexa provides a useful benchmark for any site in a similar industry, with many webmaster using the results.
crisscross (love your nick),
You are basing your belief on several statements that are not grounded in fact. This is called faith, and your beliefs are founded on this faith.
I am not talking about faith based beliefs, I am talking about the verified truth. Everything I stated in Post# 5 is verifiable and based on fact. This makes it true.
For instance, you state that your total page views are higher than the site that occupies number 4. This is not a fact, it is an assumption. Unless you have been digging around in site #4's raw logs, you DO NOT KNOW THEIR PAGE VIEWS. That is an assumption, not a fact.
Total page views: Are you getting this from your logs, or is this the aggregated data from Alexa? If it's from Alexa, you are looking at ESTIMATES. They are an extrapolation made from the sampling of Alexa users. That is not a fact.
The fact that a competitor's site appears static has more to do with them not using the toolbar. If their web site is at position #4, and yours is buried two pages later, it is a safe bet that THEY are the ones with more traffic. A safe bet. I didn't say it was a fact.
You state that your rankings went up from "30,000 to around 6,000. Our competitors have stayed static because they have not changed their sites." Alexa states:
Sudden artificial changes in the Alexa traffic rankings, from changes in hompepages and domains. Hmm... Like from a ranking of 30k to 6k after a web site consolidation? You can read it at Alexa [pages.alexa.com].
|In some cases traffic data may also be adversely affected by our "site" definitions. With tens of millions of hosts on the Internet, our automated procedures for determining which hosts are serving the "same" content may be incorrect and/or out-of-date. Similarly, [b]the determinations of domains and home pages may not always be accurate. When these determinations change (as they do periodically), there may be sudden artificial changes in the Alexa traffic rankings for some sites as a consequence. |
They further state"
|. Generally, Traffic Rankings of 100,000+ should be regarded as not reliable because the amount of data we receive is not statistically significant. Conversely, the more traffic a site receives (the closer it gets to the number 1 position), the more reliable its Traffic Ranking becomes. |
Huh? Did you read post#5 and follow the link to the top 20 Alexa web sites? If that is a reliable sampling, then up now equals down, and white is black.
You state, " It makes sense to them becaue it is independent and can be more easily understood than the methods you mention." Sorry, there is nothing more independent, or accurate for that matter, than your own traffic logs for measuring traffic.
My own methods were not about measuring traffic, as you stated. My own methods that you are referring to were about measuring the efficacy of optimization efforts by reviewing and tracking your ACTUAL KEYWORD POSITION STATISTICS.
"You may not like it, but Alexa provides a useful benchmark"
This is nothing personal to me. I neither like it or dislike it. You are the one who is in love. My feelings toward this are quite detached. I am only talking about the facts.
Benchmark? Of what? Traffic? No. The above quote ably demonstrates the unreliability of Alexa's data, and it's uselessness as a benchmark. Alexa says as much in the quote above.
Everyone likes a good review and every pro aims at not having any negative and nothing is worse than a negative review which is not even true.
Am I mistaken or are there not cases in the past where reviews of companies, and establishments such as restaurants etc have sued a website because of bad reviews they were hosting which in fact were discovered to be totally untruthful?
I am sure I seem to recall such a thing otherwise it is to easy to slander a company site or anything.
We know what our competition's page views are because they publish them. In fact, they have told us face-to-face. We are higher, yet it is not reflected in Google (or other search engines) due to slow indexing of changes in our domain name over the past six months.
We do not use the toolbar to boost rankings. The rise in our ranking has been due to the consolidation of domains and the addition of new services. This rise was not sudden, but has taken place over the last four or so months.
You seem to think that Alexa is unreliable because it has many Korean sites. The relative ranking of Korean sites is only useful for Korean sites against each other. Other sites within a certain category can use Alexa to track popularity against their competitors. For example, UK computer games sites can use the stats to measure against each other.
I suspect that Alexa will create filters for non-English language sites soon.
As you pointed out, Alexa states that sites with less than 100,000 ranking are unreliable and that sampling more results means more accuracy. Our ranking is now 6,000.
Perhaps tracking your keywords in Search Engines is a good method of showing your popularity against other sites. It seems a bit abstract for most our clients, readers and staff.
My hope is that Google will create a similar system to Alexa so that we can track site rankings on a more frequent basis than the current system.
|Made In Sheffield|
I posted this in another thread but resposting it here as it's relevant to the discussion:
My domain name (xxx.com) is used for my Web site (www.xxx.com obviously). My PC is a sub-domain of it (spud.xxx.com) on which I run Apache in order to test my site and others before going live.
Nobody else can get access to spud.xxx.com because of a firewall on my pc.
My current Alexa ranking is 116,364. If I look at "more site info" from the toolbar and then "See traffic detail" it shows that the traffic ranking is made up of:
spud.xxx.com ~ 53%
xxx.com ~ 45%
proofs.xxx.com ~ 2%
(proofs is another web server I run)
I would say that the above percentage values pretty much reflect my usage of the 3 sub-domains, so basically I have a traffic ranking of 116,364 through my use alone of Internet Explorer to view these sites (adminttedly I use it a lot). One person.
This begs the question, at what number does the Traffic Ranking actually show a significant number of people are using the site/domain? At what number is it actually a useful statistic? Obviously the difference between 100,000 and 2,000,000 can be just one person which isn't much (any) use for reference.
And shouldn't Alexa be making sure that there is a maximum number of hits per machine per day? Or that it is based on a maximum of one per day counting towards ranking per machiine? If they really want to provide useful traffic rankings I would say yes.
Currently it doesn't seem that it works in this way.
Anyway, back to the thread topic:
I have mixed feelings about the review system. I've seen a few bad reviews on sites that were unjust and understand those that feel that way too.
On the other hand, it's a pretty slick system that I'm surprised no one else has put together. The reviews don't make any difference to me when I visit a site, but I can see how it would be nice 'noise' for alexa to add to the pages. It's a pretty good promotion tactic.
I keep hearing that Alexa is artificially skewed toward Korean sites.
=== I don't think so ===
I would argue that it isn't skewed at all, and in fact, it is an indication that Alexa's rankings are better than you think.
Daum.net claims 400 million pages a day. [Read that again...]
Google claims to have 150 million search queries per day.
That's a Korean site, getting almost three times as many PV per day compared to Google. Just because you never heard of Daum.net, doesn't mean it isn't one of the biggest sites in the world, behind only Yahoo and MSN. Korea is huge on the Internet. The fact that Alexa picks up on that fact is a GOOD thing, not bad.
Regarding the reviews... yep. It is pretty easy to pick out the webmaster postings... both the good and the bad. But this only seems to happen on the smaller traffic sites. Larger sites almost always seem to have a bunch of balanced reviews and none of the webmaster spam.
And, now on to the spyware thing. Alexa captures what you enter into online forms because it usually ends up in your URL! Give Alexa credit for disclosing it. Shame on Google Toolbar for not disclosing the same thing.
<I think the ia-archiver is the Archive.org bot and alexibot is the Alexa bot. Is that correct? >
My information is that IA Archiver is the Alexa bot.
When you click the "Traffic Details" then the "Learn more about Traffic..." hyperlinks Alexa tells you the results are skewed because they only reflect users who use the alexa toolbar to do a search. They claim millions of users but no statistics beyond this superlative. A marginally useful tool but one that hardly counts.
Keep in mind that Korea's (and various other asian nations) internet access is second to none, and makes US access statistics look downright abysimal. With the information gleaned from Google and Amazon.com as well, this would indeed lead to a high number of stats coming from those areas. Korea happens to have a very high number of Internet Cafes for instance. Many many times more than in the US. Larger broadband penetration and more. At least, that's the info included in numerous articles about internet cafes, regulation, etc that have surfaced on places such as The Register, Wired and elsewhere.
I'd thus expect higher Alexa/Amazon/Google penetration. This, in my opinion, would mean that US sites with high rankings *thus globally* have "done a lot more" to have such rankings, since most people do not realize that Alexa rankings are global and not US based.
I am very familiar with Korea's high rate of broadband adoption, etc. However, that has nothing to do with my statement and is irrelevant to the point I was trying to make.
The population of South Korea is 48 million.
The population of the United States that is online is 143 million Americans in September 2001. (Per the Cyberatlas [cyberatlas.internet.com])
That figure represents almost THREE TIMES the TOTAL population of Korea. So, FoodPlaces, this is what I mean when I say that Korea's high use of the internet is irrelevant to this topic.
What is relevant is that among this minority of internet users located in South Korea, a great many happen to use the Alexa toolbar.
The implication here is, because the majority of Alexa toolbar users live in South Korea, the Alexa toolbar is a better gauge of internet traffic in South Korea, than it is in the United States. This is a Fact. The truth.
Yet some in the United States claim that the Alexa rankings have relevance to them, despite the FACT that the rankings are overwhelmingly relevant to the Korean market. Just consider the Korean dominance in the Alexa rankings. How hard is this to understand?
I think the problem here is that these Americans have an emotional attachment to their toolbars, and can't cope with the truth.
Reminds me of those Jerry Springer shows when the abused husband still wants his philandering wife back: A disconnect between reality and fiction.
As I told the other person in this thread, I neither like or dislike the Alexa toolbar. You are the one who is in love. How else to explain someone's infatuation with a "ranking" product that has no statistical relevance to their market?
Also, consider this: Alexa claims [pages.alexa.com] to have indexed 2 billion web pages. Alexa states that any ranking above 100,000 is statistically unreliable. So, 2 billion minus 100,000 = 1,999,900,000 web sites that cannot be accurately counted. Put another way, Alexa claims to accurately rank .005% of the known internet. What that means is that 99.995% of the internet cannot be accurately ranked.
Now, with those facts understood, can you still honestly believe that out of the .005% that is "accurately" ranked (for Korea, mind you), do you truly believe it when Alexa tells you that you rank number 6,000 on the entire planet? Do you seriously believe that?
The issue isn't really the accuracy of Alexa's methodology, per se. Alexa's top 500 could indeed fairly accurately represent the true ranking of the internet as viewed by the people of Korea.
The issue here is the accuracy of Alexa's rankings as it relates to webmasters in the United States. The facts demonstrate that those rankings are statistically irrelevant.
The Korean sites are only relevant when you count the ABSOLUTE global ranking. If you take out all the Korean sites, the RELATIVE rankings of English sites will still be the same!
Alexa rankings relevant for sites when you compare with other sites in your industry. I am not in competition with Korean sites. My competitors site ranking (not Korean) has been 10,000 or so for the past year. We moved from 30,000 to 6,000 or so. That is useful information to us and has nothing at all to do with Korean sites. (FYI The vast majority of Koreans do not speak English and surf domestic sites only.)
According to your logic our site is actually ranked higher in the English-only web if you take out the Korean sites.
(Although it is not necessary for RELATIVE rankings, I previously suggested Alexa make an English-sites only ranking to clear up this issue.)
The next stage is that I want our site to be ranked at the same range as other sites in the 1,000 ranking level.
When we do that we will have be able to say to our customers "We are ranked the same as _insert benchmark here_" This has an easy-to-understand PR effect.
Yesterday I realized that our site is around the same level as some larger US magazines. That is a huge boost to our confidence. How would we find out this information otherwise? for free?
Your other point is correct: Alexa reflects that most sites on the net do not have a lot of traffic.
Because you do not think it is relevant does not mean it does not have value.
|How would we find out this information otherwise? for free? |
I've not only worked for a major online magazine, I've advertised with some (quite expensive), and inquired with other sites about their site traffic. Their advertising departments will be more than glad to share this information with you. For free.
I encourage you to do so. I think you will be surprised to see what their real traffic is, as measured by actual hits to their servers.
Anyway, crisscross, I've enjoyed our dialogue. It's good to exchange information.
Sometimes people can't agree, and I think this is one of those moments. No hard feelings. I think we have to be gentlemen and agree to disagree.
Reviews are reviews.. when you look at them its pretty obvious that most have self interest motive.
Koreans do have a smaller population than the US, but their internet use penetration is much higher, especially for broadband, so high ratings for Korean sites make some sense. on the other hand, remember very few Koreans speak English, in fact hardly any, so they have less sites to choose from. The use of second languages as is common in many Europoean countries and other Asian countries is also less, so the options for Koreans are less, while others can browse sites in several languages.
Where comparing sites based on relative rankings can hit a snag is where one site has more interest to Koreans than others. For example, one of our sites which has more Korean content ranks well, another bigger and more popular site with no Korean content does not.
My rough theory is that Alexa toolbar users are not very representative of internet users worldwide. Those who use it tend to be:
1. Those with fast or braodband connections that dont mind the extra time required to download each page as Alexa calls home to collect a lot of data each time.
2. Those with good memory in their machines. Alexa occassionally crashes by 64 MB laptop.
3. Those with more than a jo blow user interest in the net who just uses like a regular user. There would be more webmasters, people with high internet skills, and those with a vested interest in the web and web promotion.
4. Koreans, who like Korean language sites and very few others, are quite sophisticated in IT skills (ie lots of tech heads!), and hard core gamers (Koreans love online gaming! - just look at the number of cybercafes in Korea and what they are doing in there!)
So like any data, its important to assess the data in terms of the extent and type of biases in the raw data and how its collected.
Representative valid data IS almost impossible to collect, and when it is, it costs HEAPS of money. As soon as Alexa data becomes substantially useful they will charge for it - or some of it. Until then, its interesting, but im not going to base my strategy and invested money on it! And i will wont to know also all the figures on where the data is coming from, and full demographics on the toolbars usage (which Alexa is collecting of course, but is not telling us!)
"Spamming" IS an issue. Alexa itself encourages webmasters to sell on, by inviting people to download the toolbar from their site. The unsaid but well underdtood nudge-nudge wink-wink implication is if your regular visitors use the toolbar you will get quite a BIG boost in the rankings. Since i rediscovered alexa from here, and istalled the tool bar just on my machine, not other staff members, we have imporved our ranking on our test site from 300,000 to around 50,000. Thats a big change for just one tool bar on one machine! Imagine if we also enlisted all our mates to do it as well...