Forum Moderators: open
Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.
At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.
The index-page was always in the top-3 position for our main topics, aka keywords.
I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index
As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.
My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.
Lets call them:
www.foo1.net and www.foo2.net
Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)
The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window
Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).
Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.
So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.
Fetched the page, but again got a code 200 with no redirects at all.
Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"
Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.
Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html
Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.
None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo
I have send an email to Google about this and am still waiting for a responds.
Here seems to be the major culprit for redirecting website traffic
There was an new way of trandferring domains implemented Friday of last week allowinf automatic transfers of domain names .
Registrant:
<snip>
I did a header server check and sure enough 302
Location: http://www.example.com/rest-of-url
Content-Length: 130
I have reported this to google.
Clint
[edited by: ciml at 10:27 am (utc) on Nov. 30, 2004]
[edit reason] No specifics please. [/edit]
When did this new and imprpoved handling of 301/302 redirects across different domains went into effect?
Google's cache displays "as retrieved on Nov 14, 2004" for all of the pages in the site (except the meta redirect page).
Disregard the above post as I was given the wrong domain name.
Clint
Anything new with your test results?
If you begin to see a URL only listing for the hijacker, you're making progress. But YOU'LL need to tell US how Google's doing! :)
<added>Since your listing reappeared about two weeks ago, I expect the hijacker will go URL only after another two weeks. I'd sure like to know how that progresses... </added>
from another forum:
"I heard that Matt asked pretty heavily for examples at the WebmasterWorld conference and only got one concrete example. If people want to send specifics (i.e. "site A appears to have duplicate pages from, or is doing a 301/302/whatever to site B, and Google is wrongly picking site A as canonical", with actual values for A and B), I'd be happy to hear them. Drop an email to webmaster [at] google.com with the keyword "canonicalpage" (all as one word) and I'll ask someone to collect the feedback and pass it on to an engineer. Being extra clear will help us with any feedback you send, e.g. "The correct site is somedomain.com, but if you do the query bla, you'll see that such-and-such.com shows up instead."
/////////////
it was posted on Nov 30th of this year so
Since Thanksgiving, I have sent two emails to Google at Googleguy's attention with examples and received a form reply. It would seem that whomever is reading the emails is simply not interested or simply does not care. Furthermore, the following thread was started and Googleguy has made no appearance:
[webmasterworld.com...]
They are aware there is a problem but it seems nothing is being done to fix it. And they prefer to send silly form replies. Easy way out. Meanwhile...good websites are falling out of the search index every day. I will send comments to Google once more, and hope to see something positive rather than a form reply.
[edited by: crobb305 at 7:34 am (utc) on Dec. 5, 2004]
try again with what he says as keyword...
C
with the keyword "canonicalpage" (all as one word)
I'd put it both in the subject line and in the message.... Why not?
C
anyone else? Over 300 postings in this thread. If you don't send it to them you can't complain later on ;). Doesn't have to be your site as an example.
In the above-cited thread, GoogleGuy is volunteering "no spam-related action based on these reports" (of inadvertent hi-jacks).
even if your site is spammy you will not get penalized. Basically they'll make believe they didn't see anything. No guarantees if someone reports you 2 months later though...which is normal.
