Dupe content checker - 302's - Page Jacking - Meta Refreshes

You make the call.

[Marcello]

My site, lets call it: www.widget.com, has been in Google for over 5-years, steadily growing year by year to about 85,000 pages including forums and articles achieved, with a PageRank of 6 and 8287 backlinks in Google, No spam, No funny stuff, No special SEO techniques nothing.

Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.

At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.

The index-page was always in the top-3 position for our main topics, aka keywords.

I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index

As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.

My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.

Lets call them:
www.foo1.net and www.foo2.net

Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)

The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window

Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).

Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.

So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.

Fetched the page, but again got a code 200 with no redirects at all.

Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"

Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.

Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html

Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.

None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo

I have send an email to Google about this and am still waiting for a responds.

[kwngian]

I think google will fix this problem eventually.

My was fixed and my traffic has doubled.

A big portion of the pages on my site was eithier hijacked or copied and all for the purpose of adsense. Now all these pages appear as supplement results.

[rocco]

what do you think, some speculation:
[webmasterworld.com...]

[tombola]

The least one can say is that these two elements (hijacked pages and this new update) are closely linked. :-(

[The_Hitcher]

I have tried to query this issue with Google and got the standard replies. Lately I've had to contact owners of directories to remove certain links that now seem to look like redirects due to the way Google is now listing them.

A clients site has a major index page problem and no matter what phrases you extract from his home page they do not appear to have any relevance whatsoever in the Google results (although actually in the Google machine), yet any other page comes up fine. I find myself trying to 'bodge' pages in an effort to get round a bug that Google has made no attempt to fix. Effectively you can take out another site because of this and I'm amazed that Google still hasn't addressed this problem.

Is their algo now so complex that even they don't understand it. This is all related to 'missing index pages', redirects and site hijacking - the issue comes up time and time again and no end in sight.

[renee]

is there a way of detecting if a request for my page is a redirect? through a script(php)?

[zeus]

renee - just make a site:yourdomain.com see if there a reother domains listed and check them out.

[Seo1]

Hi There

Doing site:yourwebsite.com does not show duplicate content, it shows results that mention your website name. That can come from any forum, community, blog, irc chat you have ever joined or written in about your website or that anyone else has written about your website and included the full url as a link.

Also I did a search on google for keyword term - ranking relevancy- and the same article was brought up several times in the first three pages. All on different sites. I don't see there being a dupe content ban with this.

Perhaps people are confused as to what is a hijacked site and one that is linked improperly or simply mentioned several times on other websites.

Clint

[renee]

zeus

thanks for the response. I'm asking though if there is a way of detecting it programmatically when the page is requested. something like the variable HTTP_REFERER yields the page that originated the request (and not the redirecting page!)

[crobb305]

Doing site:yourwebsite.com does not show duplicate content, it shows results that mention your website name

Not true. Site: command is supposed to show pages associated with your site. If other domains are showing as they do for my site: searches then those are being incorrectly tied to your domain.

[instinct]

First we google-bomb the phrase "faulty search engine" to point at a page outlining G's problems (hijacking, sandbox etc) and THEN we send out the press release.

Embarass them into action!

(Only joking)

;-)

[dazzlindonna]

Bloggers have recently had a lot of power in gaining the attention of the masses. Perhaps a concerted effort of blogging about the problem is due. I have blogged about it in the past, and I blogged about it again today. Those who have blogs, consider doing the same.

[crobb305]

I have a friend in the media who says this would certainly be an interesting news topic. I hate to suggest a press release because Google may be working on this problem. The issue is that they are remaining silent and that they have had this problem since 2003.

And legitimate businesses are suffering at the hands of malicious hijackers.

[siteseo]

As I posted in another thread, I believe part of the duty of the Supplemental Index is to weed out dupe pages/hijackers. The definition G gives for the SI is:
"Supplemental results are triggered on a relatively small number of queries for which Google's main index does not provide many results. Because this index is still in testing..."

So clearly it is still being tested.

I can only say that many pages that duplicate my content (especially affiliate links) have only recently been moved to the SI.

[crobb305]

The fact that the site:mysite.com command is still showing spammy domains being associated with my site tells me that nothing is changed. Yes, they are in the supplemental basket, but they are still incorrectly being shown as part of my site.

[walkman]

"The fact that the site:mysite.com command is still showing spammy domains being associated with my site tells me that nothing is changed. Yes, they are in the supplemental basket, but they are still incorrectly being shown as part of my site."

agree with you. It doesn't matter where they are. They were on the supplementals before I deleted them but I was still penalized. The supp. links should've been ingored, instead I got penalized too.

I still am penalized but hopefully Google will update itself. I deleted some links a week ago and the last one 3 days ago. The people who inadvertently linked with Meta or 302 have been very cooperative and nice.