Google "Custom Search" ranks my site normally, but not regular search?

[flicky]

Anyone ever seen this?

If you think you have a penalty with Google, perform a query with Google Custom Search and see if the results are the same.

Doing ANY search for my keywords in ANY Google "custom" search shows results as I would expect. However, the same search in regular Google serps is missing. It's been like this for several months.

I do know that webmasters that host Google Custom Search have control over what they want to display, however, my problem is reflected with ANY Google Custom Search instance. It's a global thing.

Any insight would be appreciated.

I'm thinking some sort of algorithmic penalty might be in place on my site and CUSTOM search does not reflect penalties of this sort?

What is Google Custom Search you ask?
[google.com...]

thanks,
marc

[tedster]

That is a very interesting discovery, flicky. Thanks much. It seems clear that some ranking ranking changes are applied to the "raw" data at the very last instant.

[flicky]

I was hoping someone else that thinks they have a penalty would test this theory...

[tedster]

How do you recommend someone create a Custom Search so they can test this idea?

[OMZen]

Not many people are aware of the fact that Google maintains a separate "special CSE index" ( to cater to their "on-demand-indexing" facility ) apart from their main index.

[googlecustomsearch.blogspot.com...]

[flicky]

If you want to test the idea that algorithmic penalties may not always be applied to the Google Custom Search index, please use the following sample site I created:

[google.com...]

If you want to create your own, follow the simple steps below...

Go to:
[google.com...]

In the box for "sites to search" just put some random site like cnn.com

Finish up the rest of the steps.

When done, you need to go the main interface and choose "Basics" from the links in the left menu. Scroll down and select "Search the entire web but emphasize included sites." under preferences. That's it...

Let me know your results.

marc

[tedster]

For experimental purposes, we asked flicky to create a test CSE and post the link. Since CNN is a Google partner, it's a good choice for weighting. If you see CNN ranking higher on some queries than normal, that would be the reason.

I've made a few test searches and so far the results agree with what I see on regular Google (minus the universal search stuff). Anyone want to test a suspected penalized word or filtered word and let us know if the results line up or not?

[tedster]

I'm really surprised that no one has any comment on this. It's very interesting to me that sites seemed to be ranked without certain filters or without certain penalties on a CSE. It could be a good diagnostic tool.

Imagine you had a #5 ranking and it fell to #15. How would you know if the algo just changed the way you are ranked, or if some kind of a penalty/filter was applied? If things are the way they seem to be, you could check your ranking on a wide ranging CSE like this - and see if you are at #5 (penalty/filter) or #15 (ranking change.)

Flicky's test CSE [google.com]

[indyank]

I too see my pre-panda ranking in that CSE. This is definitely an interesting find.

But how are you people concluding that it is the algorithmic penalties which are not reflected in the CSE index and not the manual penalties?

Also note that the pre-panda results are still available on google.com and it is just the U.S serps where the panda is dancing.

[wickedboston]

Some queries certainly have different results which do not appear to be solely due to a laten server update - however I do not believe results are different because of penalties. I could not find any results for the companies which were recently publicly penalized - could you?

One thing I noticed that was interesting was that the CSE results show multiple listings for the same domain but in Google they only showed one result - indent listings are definitely being used less. It is also interesting to note that Google put the internal page in their results over the homepage, even-though the homepage was ranked higher in CSE. The internal page was moved to the position of the homepage and the homepage was completely removed from the results.

Try a search for 'dresses' almost all the results are homepages in CSE but not in Google..

[helpnow]

Hmm... using the flicky cse, I see results from about 2 weeks POST-panda.

Definitely not PRE-panda.

And definitely not today's regaulr google serps results either. I'd say these serps are dated about March 15 or so.

[dazzlindonna]

Hm, it seemed to be pre-Panda a few days ago, but now...nope. Not from what I see.

[aish1108]

Wow, so my website really is penalized. (duh!)

I just took a unique phrase (well it used to be unique, now it's on ask.com and reference.com when I serach for it on Google) that's on every page of my 4 year old, 300 page website and on regular Google search my website shows up on the 3rd page.

On the test CSE from flicky, my website shows up for 69 of the 70 results returned except number 1! (Number 1 is a malware/spam site which returns a 404).

It's a small site and I haven't worried too much about the 80% drop in Google traffic since November 2010. Right after it happened I posted on the Google webmaster forum and got no response and sent in 2 reconsideration requests both of which were auto responded. I had nothing to admit, because I still don't know what policy the website was violating.

Since then the website returned to it's pre-November traffic for 2 periods of 4 days, once in January and once in March before shooting back down to 0 traffic from Google.

Well at least I know the website is penalized :-) Any ideas on what to do next?

[amarev]

I've tested a couple of panda penalised sites and I see no difference in the ranking *order* between the CSE index and the main index. Obviously, there are less sites in the CSE index, but the order of the results is the same.

Anybody else found this?

[PPC_Chris]

On the test site, I don't see our pre-Panda rankings.

[Planet13]

Well, I see a definite time lag. The results I see are from either a month prior to today, or long before that.

Another thing I see is that several of the top four sites have multiple pages listed throughout the top four or five SERPs. In the official google results, this has been trimmed down, so most of those top ranking pages now only have about three or four pages (while before it was more like 6 to 10 pages).