Google's site: query returns bad results - now being fixed

hyphenated domains and trailing slash get incorrect results

[Adam_Lasnik]

Hey everyone,

My colleague Vanessa just posted a note which should hopefully make your weekend a less stressful one :).

Specifically, she's noted that we've had some issues with our "site:" operator [sitemaps.blogspot.com] not returning appropriate results when used with hyphenated domains... so it's quite likely that much of the falloff you've seen is thankfully illusory.

We've noted similar discrepancies with site: searches using domains with a trailing slash as well.

And yes, Googlers are working to correct this stuff as quickly as possible!

[MLHmptn]

All of this certainly makes you want to ban Google. Nothing is more perturbing than following Google's quality guidelines and getting this in return. One would begin to wonder if Google has been using their Toolbar to rank sites kind of like Alexa has been doing for years. If the toolbar doesn't rank you within the top 50,000 sites your site doesn't exist beyond the home page. Just a thought as it seems if your site is not highly valued your entire site is now supplemental excluding home page. All of this is beginning to make me wonder if we are going to have to have an authority site pointing to every page on our sites just for indexing purposes. Hopefully this update or whatever you want to call it isn't going to backfire on Google.

ON another note I still don't belive this is a hyphenated domain issue either as a few sites I monitor have 0 hypen's in their domain and yet they are 99.99% supplemental now.

[edited by: MLHmptn at 5:15 am (utc) on May 24, 2006]

[Jesse_Smith]

It looks like this has nothing to do with the hyphen, and it looks like it's not even close to being fixed.

I got one domain that was made this year and it has over 200,000 pages indexed, and has two hyphens in the domain.

I got a nine year old domain that has 189 URLs indexed, with two hyphens, and I got a domain with no hyphen that has 348 results, another domain with no hyphen that has a measly 10 URLs indexed, another domain with no hyphen that has a measly two URLs indexed, another domain....

Just like I got one hyphen domain that wasn't touched from this, I got one non-hypen domain that also wasn't touched, that has 23,700 URLs indexed.

[Dayo_UK]

Hmmmz - I wonder.

This information coming out of Google that it is just the site:domain.com operator just does not seem to add up.

I have a site that site:domain.com shows about 4-5,000 pages and site:domain.com/ shows 1 page (as per stated would happen by Google)

How about allinurl:domain.com - only shows one page.

So this site clearly only has one page indexed to me - sometimes supplementals can be pulled up - but most of the time they are not being displayed.

So I dont think that is an issue with showing results in a site:domain.com search rather an issue with indexing and pulling data from the database as g1smd mentioned.

But where are they failing - it is not just the hyphen - it is also the / ....... well every single page except the homepage for sites has the / in the url - with another thread saying that Google is going only a couple of levels deep - perhaps due to PR - but perhaps it is to do with the url structure - eg Google is getting confused in indexing page with to many /// and to many -----?

[fred9989]

As with everything else in this sequence of changes, there seem to be as many contradictory reports of hyphenated domains being affected as
not affected.
A couple of simple observations about Google:
1) many (say 60+%) of my hyphenated domains fail to appear in response to searching for
my-domain.com
but are still in the index, and do appear with a site:my-domain.com
command, albeit with fewer pages. They also appear on a search for specific unique text extracts, so they are indexed. However, they do not appear in response to relevant search terms in the way they once did.
2) The other 40% are behaving normally. They probably have higher PR and more inbound links of better quality. Maybe.
3) A search in my areas of interest reveals very few results with hyphenated domains in them.
Conclusions?
None really, apart from banal generalities like Google has a bug. But could it be that all of this results from some attempt to check spam by downgrading a hyphenated domain unless it has high PR?
If so, and the engineers have been playing with the code to check this out, maybe they cannot now fix it since it is so complex it's growing out of control.
I think I just said "Google has a bug". Oh well.
Rod

[suthra]

a search for site:<ourdomain.com> now shows only 500 Results, there was more than 28K pages in google index last week .

Also 2 of our new portals appears and disappears at frequent intervals.

[gendude]

>> Nothing is more perturbing than following Google's quality guidelines and getting this in return.

That's what has myself and some of my competition in one niche scratching our heads and getting frustrated with google - we looked over each others' sites with a critical eye, couldn't find anything that is amiss (mine is hyphenated, the other two competing with me are not). Yet the spammers are doing better than all three of us in keyword and site: listings.

>> One would begin to wonder if Google has been using their Toolbar to rank sites kind of like Alexa has been doing for years.

I can't imagine they would - if they resorted to that, after giving us all these guidelines over the past few years, then they are basically telling us that they (the guidelines) are meaningless, and we might as well resort to blackhat SEO.

[g1smd]

If you make a search for a hyphenated email address and then replace the hyphen with a space and search again, the new search brings up a lot more results: all of them Supplemental results that do contain the email address.

This is not just a site: operator problem.

[trinorthlighting]

My supplemental results are not showing up for the site:mysite.com

They are showing up in certain keyword searches for my niche though.

[chris41]

I have the same problem.

I am No 1 in google for al least 3 2 word keywords and in top 10 for mayn others in this domain.

Yahoo and msn returns 100 pages linking back to my site.

Yahoo and msn return 1000+ pages in their index.

Google returned tha last months 2 in index o backlinks.

After they made the post I tried without the trailing slash and nothing came up then I tried this:

site:[spaces]example.com amd then i got some pages back.

It is realy annoying....

[tedster]

If you put the space after site: then you are no longer using the Google site operator -- you are doing a regular search on two terms - site and example.com

[sett]

Use "site:" command in Google for "DMOZ"

Serch with site:dmoz.org (Returns 26 million results)

Serch with site:www.dmoz.org (Returns 0 results)

Is site: is also not working properly for "dmoz"?

[Petra Kaiser]

For a search on site:domain.com/pagename.html I get 2 results for the same page with different cache! I never recognized this before.
DC=64.233.183.104 cache is May 23 2006 and May 15 2006

[F_Rose]

Adam and Venessa with Google posted the bug should be fixed within a couple of days.

Haven't seen anything yet..

Looks like the problem is a lot more complicated.

[gendude]

>> Adam and Venessa with Google posted the bug should be fixed within a couple of days.

How many days ago was that?

This is more serious than they are letting on.

[JuniorOptimizer]

Yeah, this is not fixed, and there's be no communication after the first post.

Funny thing is, the bot is insatiable. In order to achieve a 9 page count, Googlebot has accessed the site 33693 times in the last 4 days. I think someone else said it best The Madness of King Google is now very noticable.

[kamran mohammed]

>> Adam and Venessa with Google posted the bug should be fixed within a couple of days.
How many days ago was that?
This is more serious than they are letting on.

Venessa made a post on 19th May - read the post below.
[sitemaps.blogspot.com...]

[F_Rose]

Adam has responded to an e-mail I sent regarding our site.

It reads as follows:

"I've verified that we haven't assessed any penalties against your site, but I do understand and respect your concern about the lack of results shown for site:www.sample-e.com/

Please note that we're currently dealing with a known bug now which skews such display:
[sitemaps.blogspot.com...]

It is very likely, therefore, that your pages are still visible in our index, just not with this operator used.

If you've seen an appreciable drop in traffic from Google, however, please let me know and I'll ask someone to look into the matter.

Regards,
Adam, on behalf of the Search Quality Team"

[kamran mohammed]

F_Rose, Thanks for the post with the update from Adam.

But as Adam has asked about the traffic issue, Yes that's True that the Traffic has gone down miserably. I hope what he says is true that he can see the pages indexed at his end but we can see it with the help of Site:operator.

I wish they fix the problem at the earliest.

But I have noticed one more thing in the past. I have seen that this update was rolled out on 17th May and Matt has posted just one day before the update on his blog about the timeline of Big Daddy Data Center and just after 2 days Venessa posted on her blog about the site operator malfunction.

I was wondering why did Matt mentioned about it one day before and why is Matt on vacation when everyone needs his feedback and Adam is all lost.

Did anyone notice all these things. Now I am quite skeptical to how things are taking shape at Google.

What does others has to say about it?

[JuniorOptimizer]

kamran mohammed,

I share your skepticism.

[kamran mohammed]

Sett said -

Use "site:" command in Google for "DMOZ"
Serch with site:dmoz.org (Returns 26 million results)
Serch with site:www.dmoz.org (Returns 0 results)
Is site: is also not working properly for "dmoz"?

Has anyone checked what Sett has posted here. That's a good observation Sett.

Now what does this mean that DMOZ is also not showing with Site:operator

Things looks more worst from this end but no one at Plex is bothered about the madness of website owners at the moment.

And I have noticed one thing in the past most of the time - that whenever there is an update on Google why do we see the snippet description from DMOZ? Has anyone else noticed this before?

But why only description?

Well that's my observation.

[g1smd]

>> "....fixed in a few days...."

...and six days later there is no change visibile.

I think this will take them several or many weeks to fix.

[g1smd]

>> Search with site:www.dmoz.org (Returns 0 results) <<

The ODP site is all indexed at dmoz.org so there should be zero pages shown for site:www.dmoz.org - so that search IS correct.

However, maybe you should check my forum posts from (about) a year ago when "302 redirect URL hijacks" were being discussed here.

At that time, I observed that a site:dmoz.org search showed a list of the regular ODP pages, and that site:www.dmoz.org showed tens of millions of results, none of which were from the dmoz.org site itself.

In fact all of the site:www.dmoz.org results showed sites that had done a 302 hijack on ODP pages. It was about the most blatent example that I could find on the web.

It was like this for many months.

Within 24 hours of mentioning it here at WebmasterWorld the site:www.dmoz.org search showed zero results and has done ever since.

The other examples that I know about, and did not mention here mostly remained unfixed for very many more months after that.

Draw your own conclusions as to what happened, but I call "hand edited result" on that one.

[edited by: g1smd at 9:33 pm (utc) on May 25, 2006]

[1984bb]

it shows when you do
site:dmoz.org

[JuniorOptimizer]

I think this will take them several or many weeks to fix.

I totally agree. And the problem with that? Weeks really add up! We're talking about a chunk of time and time is money.

I know Google doesn't owe us anything...and they sure know it too :)

[Frederic1]

Google has not fixed the bug yet? I'm beginning to think that it's not just a "site:" bug but it may also involve search results :( (pages dropping). I fear it's a serious bug that they are fixing.

[Jesse_Smith]

Check your sites in the datacenters. I think it's *almost* fixed.

For example, for one site, I got '27 results' on Google.com and '133,000 results' on the datacenters.

It just hasn't made it over to Google.com.

[arubicus]

Ours look way inflated but it is a change.

[Petra Kaiser]

This morning Google Sitemaps shows me a 404 HTTP-error for the page: domain.com/1.html calculated on May 22
We do not have pages named 1.html.
But there is a link to a page like “whatever-1.html” on a page cached on May 21
I checked internal and external linking, no links to 1.html.
No further comments!

[avalanche101]

1.html site maps error,

We've got that one as well.
Initially thought we had a duff link on a page to a www.site.com/1.html.

After laborious checking, guess what - no link to a 1.html on any of our pages and certainly no 1.html.

What is going on?

[F_Rose]

"Check your sites in the datacenters. I think it's *almost* fixed."

Which datacenter have you tried?