Forum Moderators: mack
Steveb, this would be one of those cases where I couldn't agree with you more. I think what's happening is this, more or less:
most users have learned how to do search to some degree. You type in a few words, you get results. If you generally get the results you're looking for, you keep the search engine. If you start not getting those results, and hear of a new one from a friend, with a catchy name, say 'google' for instance, you check it out. If the search results seem better, you slowly stop using your old search engine and switch to the new one.
This is how google achieved their market dominance. But then something happens in this process. Search just isn't that big of a game in any larger sense, it's just some keywords that people would like a relevant answer to. Main focus needs to be on getting rid of spam, not all websites from the last year.
So what do search companies try to do? They think, hey, we'll try to make our results 'better' by 'guessing' at 'average user' desires. In order to get or maintain their market share, but a market share of what is essentially an unbranded, commodity item. Show me a user who has deep emotional attachment to google and I'll show you a member of WebmasterWorld who posts all the time on the google forums. Nobody else cares.
But all the searcher really needs is an answer to a small question, here's some keywords, give me the right pages in the top 10. Tweak this too much, or lose focus on this central element, and google becomes altavista, which I think it may be doing at this very moment, at least they are opening themselves up to the next search engine company to come along that builds on the idea of delivering relevant results from such keyword searches, but focuses on the real problem, which is developing an affective spam blocking engine/human review system. That search engine will win, not the search engine that guesses what locale you're in, guesses that you are looking for something in your locale, etc. That's stupid, like most new search ideas, like semantic indexing, which is downright retarded, in mikec's eloquent terminology.
Patrol the top 5 pages for the top 50 spammy cat's and weed out the obvious crap. Have 10 intern's doing this 5 days a week and you'd see a much better engine, be it Yahoo, MSN, or Goofle. Checking backlinks and the things we all do in researching why someones ahead of us.
With the profits being what they are, surely this is a realistic expense in dealing with spam.
Go figure.
This means a few things: first, and most important, programmers cannot be the primary decision makers, because programmer's have really big egos that think everything can be programmed if given the right people and team leaders.
This is simply wrong. I came to realize this when I started playing with advanced email spam filtering tools, which select suspect emails out of all incoming emails and present them to you as suspect. Once you tell it what was spam it modifies how it looks. After about one month one test gave I think 1 false positive result out of something like 10,000 emails.
The human brain is better at this game than the programmer's attempt to model that human brain, that's the way it is, and will be for the foreseeable future. Change the pattern slightly and the program fails to recognize it, that's the same exact problem all voice recognition currently has, it needs either very simple sounds, unique words [thing keywords], or it needs to be trained to your specific tone and voice. Change any part of that and it fails. That's how all spammers work, they change the patterns faster than the programmed solutions can keep up, that's why anti emaili spam depends on humans to tell it what is and is not spam, and then it builds up itself from there.
But the programmers have to learn how to put their egos in check. There is little danger of sergey and brinn, or bill gates, doing that, which means I expect a new search engine company to rise at some point.
Would suggest one modification :: interns aren't even required. You can easily find good outsourced employees for $500 or less a month. Location in this case truly doesn't matter.
CF
You know, when I was thinking about this, I was thinking about people getting paid about $8 an hour in the san francisco bay area, temp workers, which means a total of about 20 an hour altogether usually, but I didn't even think of outsourcing it. Although I think that there might be some cultural issues in outsourcing, we are so fluent in our english web culture that spotting spam takes only seconds for us, I'm not sure if outsourced results would be as reliable, at least not if my experience with outsourced call centers etc is any guide, which I think it is.
But even using temp workers we're not talking huge amounts of money here. Then if you add levels to it, like when you find certain sites that are autogenerated, they of course will always be the same html etc more or less, so you can easily add a component that puts any new site that fits that profile into a higher category, almost certainly spam, probably wouldn't even need to be human moderated except to double check now and then.
Actually, given that MS really doesn't have much attachment to any web project they do, and have no problem using temps for a lot of jobs, and taking advantage of those temps, maybe it will be msn that does this first, I don't see sergey and brinn having what it takes to admit that their way isn't working at all anymore, and then to open their eyes and start looking for non phd programmer based solutions. That takes a special kind of person, and I don't think they are that type of person. You know the type, the one who says, oh, man, I'm wrong, I'll have to change how I look at this even though the world tells me I'm great and gives me 1 billion, but I know inside my product isn't working anymore like I dreamed when I made it... aka humility etc.
This looks to me like and error since they have a tick box clearly intended for people who really want local search. Will they get around to fix/change it? Well they really should. It looks bad.
"I don't understand why one of the search engines haven't taken the combination crawling/human editing approach. "
ODP, Yahoo Directory, Looksmart...and even the curent Yahoo SERPS to a certain extent all have tried to control the web by hand with varying degrees of failure.
Google have the right idea here, pay your Geeks to perfect the Algo that way you don't have to keep manually reviewing the SERPS for ever and a day.
[beta.search.msn.com...]
co.uk does manage to also rank webmasterworld.com first, but apprently MSN thinks people in the UK want to find ciml and receptional's pages in the top five, but US people shouldn't see them in the top 100.
It is incompetent to geo locate searches like this.
Guess none of them actually care about being the "best". Shame on them. :-)
Just recently i noticed a website that features high on Yahoo using spam methods. Also, the site in question has built its entire back link structure on about 100 directories all PR0 and which the webmaster owns. The webmaster has clearly coppied the directory into 100 domain names with exact link structure for perfect Yahoo SEO for the site they are pushing and its paid off big style.
Meanwhile, im doing everything by the book building up links the hard way for my site(some links are not perfect anchor links, because you cant always get them agreed exactly as you want them) whilst being held behind dodgy sites.
The principle of Links doesnt work now if you ask me. Google and Yahoo need to change soon or MSN will take the market, its just a matter of time
Perhaps the geo-location is partially based on the content meta tag. I say this because everything else about a site should indicate UK or at least WW but the site shows up in northa merican results. HOWEVER, the content tag on the homepage is set to "en" not "en_gb". ("en_gb" means you are using British English not American English)
Relevant internal pages are "en_gb" but maybe they don't matter as the homepage determines the setup for the domain. I realise that languages don't perfectly map with regions but there is a pretty strong correlation.
If so, I'd actually like this...it allows the user to partially specify the country. Of course, like any feature, it opens the doors to spam but that's the webmasterworld we live in.
CF
We are however a Canadian company, and we are hosted in Canada. We believe that Geo-location should be based on on-page content, not what your .tld or what is even more common (determined from the IP address of your ISP provider).
If this Geo-location bias continues, we will be forced to host in the United States.
I am in Spain and even if I specify MSN.com I get forced to use the MSN.es
Not only that but the only choice offerred is
"in Spanish" or "from Spain"
(oh and the "beta" has gone)
This is crap.
I want ALL results, not be limited (forced) to only Spanish ones!
So far I've had nothing but praise for MSN for taking on google and yahoo, but this has spoiled that.
So, if I want to search for hosting companies based in the US, how do I do that. Well, I would have to switch to google or yahoo.
I think this is a bad move by MSN.