|Drop Site Match, and drop off the face of the Earth?|
Ending Site Match inclusion seems to spoil natural search results at Yahoo!
We were using Site Match through a 3rd-party SEO company. As a result, we had over 9,000 URLs indexed by Yahoo! and MSN. We just quit Site Match, and I've confirmed that none of our Site Match URLs are included anymore, which is easy to verifiy since we had a special keycode embeded in the URLs. Likewise, now we only have <300 URLs indexed by Yahoo! Why so few? Google crawls our site and indexes over 9,000 URLs!
The real problem is that our rank in search results at Yahoo! has changed drastically after we ended Site Match. For example, we have a URL that is still included in the Yahoo! index (only without the tracking keycode), which can be verified with an advanced search limited to our domain name with the file name of the page in the URL. When we were submitting this URL through Site Match, it ranked well. Now it doesn't even show up, yet there are many poorer-quality matches showing up, like other companies' product pages that aren't well optimized at all for our target keyword phrase showing up. The keyword phrase is "[snip]." There are pages with only one instance of these terms in their content, and they're showing up on the first page of results, but we're not showing up at all. ...yet our URL is in the index, because we can verify that with an advanced search.
What might explain this discrepency? Yahoo! claims the use of Site Match does not affect rank, but clearly that is not the case. Is there an avenue through Yahoo! to address this? The Site Match people gave me a very routine-sounding disclaimer that they cannot help me because we do not have an account, so they were no help at all. Could the SEO firm we were using be using some "black hat" tactics to hurt our ranking? Our site is very well optimized now, which we did in-house. How often does Yahoo! do a deep crawl of an entire site?
[edited by: pageoneresults at 3:14 pm (utc) on May 13, 2004]
[edit reason] Removed Specifics - Please Refer to TOS [/edit]
Yes, we paused our sitematch and have the same problem...shortly after we paused our campaign, our "free" listings were penalized....we went from 20,000+ pages indexed and found in SERPS to <100 with none able to be found in the SERPs.
I get the feeling that, no matter how much they deny it, Yahoo's back room strategy is to force people to use sitematch to get into the serps. When you've actually paid them with Sitematch, it might be even worse because they know they can get money out of you...so they must see you as a highly qualified lead..so if they kick you out of the free search SERPs you'll probably just pay them again to get at least some presence restored....dirty business if you ask me.
Consider this. There are 2 spiders and 2 indexes; one for those in site match, one for the organic listing. To avoid showing duplicate listing for the same page, there is filter to remove the non-site match listing. When you stop site match, the site match listing disappears, but the filter still prevents your organic listing from appearing. Eventually the filter will be updated, but why hurry if it keeping it on for some period encourages use of site match.
Just my supposition. I am still waiting for my Inktomi PFI pages to be spidered and indexed. Google seems to love them.
I hear you, but I think it's deeper than that...we only had a limited number of pages (600) on sitematch and had 23,000+ pages with organic listings...if it was just filtering out the duplicated pages from sitematch then why did all 20,000+ pages basically disappear? I've been told we are now penalized...strange coincidence. I've been emailing email@example.com to try and get specific reasons/remedies to no avail. We're approved by sitematch reviewers but penalized by organic free search...this is an inconsistency that speaks to the surreptitious nature of this whole thing.
Hmm... Already we see a few people experiencing this. Doesn't look good. Is there a Yahoo! person among us who would care to try to squelch our suspicions?
How did you learn you have a penalty. I have had many sites dropped by Yahoo recently and have no idea why. If I have a penalty at least I would know the cause.
Can you tell me how to find this out? Thank you for any assistance.
Well, the one response I did get from firstname.lastname@example.org included:
"It has been determined that your site does not comply with Yahoo!’s Content Policy Guidelines located at [help.yahoo.com...] Below are some answers to common questions regarding this issue...."
Just some general thoughts on some of the comments here:
Yahoo's organic listings are completely separate from Site Match -- and being a Site Match customer has no correlation to the frequency of crawls.
The vast majority of Yahoo's listings (99%) are, and will continue to be, from the organic crawl. This is a stat we’ve quoted many times.
"and being a Site Match customer has no correlation to the frequency of crawls."
Where did the garanteed 48 hour refresh go for Site Match?
>The vast majority of Yahoo's listings (99%) are, and will continue to be, from the organic crawl. This is a stat we’ve quoted many times.
Since most searches only look at the first page and hardly anybody goes beyond the third page, it will be much more interesting to get the statistics on what percentage of first page results in the commercial categories is from the organic crawl.
For example out of a million results search, if sitematch pages are only 20 in number and the rest 999,980 pages (99.998%) are organic but if the sitematch pages occupy the top twenty spots, probably getting 49% of the clicks closely following the 50% clicks received by top five sponsor links and many side sponsor links, it would be more accurate to say that:
Sitematch and Sponsors: 0.002% of the pages getting 99% of clicks
Organic Pages: 99.998% of the pages getting 1% of clicks
"Where did the garanteed 48 hour refresh go for Site Match?"
Without meaning to put words into Yahoo Mike's mouth (or fingers onto his keyboard?) I think that he means that if you are in Site Match, it has no impact on the frequency of the free crawl to your site.
Site Match does have the 48 hour refresh.
But Site Match does not boost ranking.
The same relvancy algorithm is applied to Site Match results and the free crawl results.
The only advtange that Site Match gives you in terms of trying to rank better is an advantage in having a quick refresh on your site, to test changes. This means you have the ability to test your SEO skills in a shorter period of time to see what the impact on ranking is - both positive and negative.
So you have consider that some people will be using this feature to improve where they rank. So logically 1% of the URLs in the index may indeed receive more than 1% of the clicks.
No conspiracy theory here - just people making sure they are getting the best out of their investment in Site Match.
I was under the impression that sitematch site often rank better as follows:
Slurp / Yahoo has few or no backlinks for the majority of sitematch pages. Thats why the webmasters in question are paying for inclusion.
To make these sites competitive with natural results and give a "level" playing field yahoo assigns some generic page rank factor.
Seeing as these are paying customers and the rest of us are loafers (and that yahoo nets some profit on each sitematch click) it is better to error in the favor of subscribers.
As far as your site disappearing:
I dont claim to know (nor do i care) how yahoo passes page rank, but let us hypothesize here. Your sitemap pages were assigned some rank greater then their natural rank, and (just perhaps) passed on some portion of this inflated rank to any pages which they linked to. Yahoo / Slurp sees the (linked) pages as important, indexes them and thanks to sitematch you have your entire site looking real pretty. In google terms you have 300 or so say pr5 pages each deep linking to say 20 of your navigation elements.
Now you quit sitematch - inflated pages are gone, natural pages have nothing pointing to them and dry up, and that handful of pages that manage to avoid this culling are a heck of a lot less important. This seems to fit rather well with what you have described.
I have not done sitematch specifically because many claim that ending the program not only kills ppi pages but affects all pages (and this is generally associated with some arbitrary timeframe) - but given something vaguely similar to the above scenario (and perhaps a gradual reindexing to satisfy said timeframe) the disappearance of ex-sitematch sites is a non-malicious and unavoidable site effect, rather then the f'in yahoo thing we make it out to be.
We were in Sitematch, and now we are not. We have seen absolutely no change in our rankings or our traffic. We continue to have new pages crawled and added to the Yahoo Index almost daily.
"I have not done sitematch specifically because many claim that ending the program not only kills ppi pages but affects all pages "
I've spoken with several webmasters that are also avoiding sitematch now because of this issue.
If Yahoo wants to avoid this negative consequence, they should make sure that the free crawled pages for sitematch customers stay in the index...word will get out that it's "ok" to use the service again. As it stands now people are getting scared.
It's just too much of a conincidence that we were in yahoo free/organic for so long UNTIL we did our sitematch campaign then paused it...then poof! we're penalized... Does Yahoo really expect me to believe that one had nothing to do with the other? It would be one thing if they could tell me exactly what the penalty was for and how I can remedy it, but I get little to no help. You would think after paying them $8,000 via sitematch that they would at least give us the courtesy of a straight answer on the issue and how to solve it...but I guess their unspoken intention in ignoring everyone is to force them into paying to get back in (via sitematch). I wish I didn't think this way, but the circumstances are too obvious not to. C'mon Yahoo, prove that I'm wrong...lifting the organic penalty for the sitematch approved sites would go a long way in restoring credibility in my book.