Forum Moderators: open
Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.
At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.
The index-page was always in the top-3 position for our main topics, aka keywords.
I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index
As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.
My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.
Lets call them:
www.foo1.net and www.foo2.net
Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)
The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window
Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).
Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.
So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.
Fetched the page, but again got a code 200 with no redirects at all.
Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"
Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.
Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html
Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.
None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo
I have send an email to Google about this and am still waiting for a responds.
You have a PR5
www.checkpagerank.com
What did you have?
How old are links to your site?
Google did an Internet Backlink Update last month and dropped PR on many sites due to stale links.
Google has also moved to continual indexing as opposed to monthly indexing.
Clint
OK this is some interesting reading, I also have lost some big ranking, so how do you see if you got hijacked and what can you do - please a biginner guide.
Search google for the following:
inurl:www.domain.com
Look for URLs with "tracker2" included in the url with a redirect to your own site similar to the following:
www.hijacker'sdomain.com/tracker2.php?url=http://www.innocentvictum'sdomain.com/subpagegoeshere.html
Then look for any other URL that has a redirect hidden in the title of the page i.e., FILE NOT FOUND, which redirects from their page to yours.
Run your mouse over every url and watch the bottom of your browser and see where it goes. If you suspect foul play click on the link to see what happens.
Then gather domain name data, contact email off the offending web site (if there is any) and write the culprit. If the email contact bounces then contact the hosting company and write them. The hosting company is listed at the bottom of the domain name data. If there is only IP numbers use this site to gain more info:
[freeality.com...]
scroll down to <b>domain name search</b> and input the IP address into:
<b>EONS reverse IP address</b> and it will give you the name and contact data of the hosting Co.
Ok I dont think I have seen that but I will have a look I have seen many with //mydomaname.com.otherdomain.com and then with text on the site with my domain name.
P.s Damn I just found some, what can they get out of that? and shall I also report it to Google
There are also redirects like this too.
theirsite.com/outhttp://mysite.com.html
People are finding all kinds of different variations of the same problem.
It does seem like google is trying to work on it because stuff is changing but as for my sites the problem is still there.
I guess it takes longer for google to update now that their database is twice as big. But we'll see what happens in the next few weeks.
Now all we need is logs proving that Googlebot behaves the same way with redirects from one domain to another.
Additional testing using redirects from one domain to another reveals:
-301 redirect test page is no longer indexed
-302 redirect test page is no longer indexed
-meta refresh test page (0 seconds delay) shows a page from before the redirect was implemented in Google's cache and the indexed URL points to the meta refresh test page
Google appears to be making good progress with this problem although it's taken a looonnng time. Sites currently hijacked can probably expect a delay of several months before hijacks are fully unindexed/reindexed and PR is fully restored.
Re the tracker2 hijacker that I reported to his hosting company (the music cd company). The host threatened to remove their site if they didn't remove the link and while it was still up yesterday (24 hours later) it's dissappeared from Google already today.
I haven't had the same luck with the other tracker2 hijacker's Host however. I'll have to climb higher up the tree I guess.
Having uploaded a huge update to my site a few days ago, I decided to do a site: search to see how things are going. Imagine my surprise when this yielded three results, the urls of which all point to other sites.All these pages use meta redirects with a two-second delay to initiate the download of files from my site.
So it would appear that Google have changed something with respect to meta redirects, but it looks like a bodge rather than a proper fix.
This does suggest that diagnosing a page-hijack by redirect is easier now - just do a site: search.
I have been too busy to follow this problem recently, so I apologise if the above has already been discussed.
Kaled.
htt p://ww wgoo gle.com/co ntact/spamreport.html
Clint
Additional testing using redirects from one domain to another reveals:
-301 redirect test page is no longer indexed
-302 redirect test page is no longer indexed
-meta refresh test page (0 seconds delay) shows a page from before the redirect was implemented in Google's cache and the indexed URL points to the meta refresh test page
Thanks for sharing these results, Dave. Promising, indeed.
Is this something you noticed recently?
When did this new and imprpoved handling of 301/302 redirects across different domains went into effect?
Thanks again for these very interesting test results.
