Forum Moderators: open
We operate several large sites and Google has indexed well over 30x what Yahoo! has. And at the rate they're adding new pages, they'll never catch up.
So, Tim, when are you guys rolling out a more competitive indexing bot or was all the talk at PubCon about Yahoo!'s commitment to indexing the ENTIRE web for free just talk?
That is a really good observation.
Google picks up new changes within a day usually.
I use a redirect on a domain because people mispell my domain name. Yahoo has this result on top when you search for my site. I changed the page to have a link and no content. This was 2 weeks ago and Yahoo hasnt picked it up.
Something like this depreciates the value of search results on Yahoo because users are taken to a page that no longer contains the content that Yahoo is showing it to have.
Added:
Cabos, I have a similar situation, just that mines been like that for more than a couple weeks.
--rant
1. I made the mistake of registering a new domain name and pointing it to another site of mine using a CNAME. No external sites to my knowledge ever linked to this (new) name.
2. My entire main site was indexed my Ink using the temporary cname intended for another site.
3. My main site is nowhere to be found under its proper domain in any of the ink engines.
4. I notice the problem almost a year ago now and implemented a perminent http header redirect.
5. Almost a year later my main site can only be found by ink search engines with the wrong name. I don't want to turn off the redirect since I get traffic from it.
6. This problem is now in the new Yahoo engine.
--/rant
Google has never once indexed the site by the wrong name. I can't be the only one with problems like this. I think it's a very poor quality spider and engine.
[edited by: peterdaly at 7:03 pm (utc) on Mar. 31, 2004]
Google has raised the bar by saying to searches, you will find really fresh content and by letting content providers know that new content will show up within a couple of days.
Yahoo is trying to sell the freshness feature as a benefit. Its almost like they feel they have monopolistic power and can force users to pay for freshness. The problem for them is they arent the only search engine in town.
As much as they are convinced that freshness can be sold, it's a feature that is a standard offering of their major competitor.
The marketplace will rule. In the short term Yahoo will see a bump up in site match revenues, as advertisers experiment, and new sites try it due to Yahoo's aggresive marketing, though over time as consumer knowledge increases, the revenues from site match will drop off.
GG said along these lines a year ago.
I think the part where we differed is on pay-for-inclusion. We've never done it because we thought it was better for users not to bring money into our editorial results; there's the danger that you create an incentive not to crawl as well so that webmasters have to pay to get crawled.
[webmasterworld.com...] (emphasis mine.)
Tweaking may work for them in the short term.
At the moment they have the benefit of a large user base. The problem is that we are all strapped for time and will go to the search engine that produces the best product/results.
The herd mentality and "buzz" about what search engine is best will dictate. Negative buzz can kill a product fast.
In a short period of time, Yahoo has created plenty of buzz. Question though whether this buzz is positive or negative?
Particularly if your main competitor is busy explaining why they think the strategy you adopted is flawed. While G's management is not directly criticizing Y's approach, they have given a number of interviews explaining their negative views of PFI and why they don't think they should go through that route. G has certainly been busy distancing themselves from PFI, and explaining why the approach they are taking is the high road to search.