Forum Moderators: open
Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.
At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.
The index-page was always in the top-3 position for our main topics, aka keywords.
I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index
As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.
My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.
Lets call them:
www.foo1.net and www.foo2.net
Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)
The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window
Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).
Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.
So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.
Fetched the page, but again got a code 200 with no redirects at all.
Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"
Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.
Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html
Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.
None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo
I have send an email to Google about this and am still waiting for a responds.
And I would appreciate a reaction from Google or an answer to the emails I have send and to Google and to Adsense
Ok its Midnight here ... i am of to bed
How about this
1) Take a snaphot of a Google-cached page crediting the copyright to the wrong site.
2) Take a snapshot of the meta redirect code (by disabling such redirects in your browser).
3) Hire a lawyer.
4) Slap Google and the cheating scumbag site jointly with a 100 billion dollar lawsuit - that would probably be the largest claim for punitive damages in history. If that doesn't attract some publicity, we can only assume Google have secretly bought the world's news media.
On the matter of which of two or more pages is original, the only valid test is which page was first indexed. However, if Google don't keep that information then it isn't possible. That being the case, duplicates should only be removed if they are on the same site.
Kaled.
shamed into fixing
This is a good point. Now that I fully understand the issues, I have got to say I'm outraged. None of my sites have been hijacked, yet, but this is something I should not have to worry about. As the news spreads, it is going to become more and more common.
Fixing this should be priority one for Google.
I recommend:
1) folks going to pubcon getting straight answers out of Google reps.
2) Bay area folks picketing the google-plex (I'm willing to join in). I'm thinking if we can get twenty or so of us down there with signs for half a day and alert the news, this problem will be fixed within a few days.
Let's take them out back the woodshed and beat the sh*t out of them!
</mob>
:)
I think this aspect frustrates me the most... because the fix to change how to handle 301 and 302 redirects is seemingly simple there *has* to be some reason why it can't just be done simply. I have seen a good number of page hijackers are selling clicks. The search companies have branched off into marketing as a primary focus it seems and maybe this is one of the results.
also, was wondering if periodically varying the wording of the index page's title might help.
I'm sure a class action lawyer wouldn't mind getting some press for this.
This is very disturbing, especially if monitary gains are being made by the people doing this.
Do no evil? huh.... sounds pretty evil to me.
