Forum Moderators: open
Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.
At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.
The index-page was always in the top-3 position for our main topics, aka keywords.
I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index
As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.
My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.
Lets call them:
www.foo1.net and www.foo2.net
Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)
The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window
Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).
Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.
So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.
Fetched the page, but again got a code 200 with no redirects at all.
Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"
Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.
Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html
Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.
None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo
I have send an email to Google about this and am still waiting for a responds.
-----
www.widgets.com - has text on homepage describing widgets and stuff. In particular there is a line: "Widgets are the new gudgeon clips."
Searching on google for that line should only find my widgets site but also listed is:
www.foo.com/all-about-widgets/gudgeon-clips.html
The google entry for foo shows my text description but clicking the link goes to the foo site which shows nothing about widgets or gudgeon clips at all.
--------
Is my site being hijacked like marcello's? How can I probe foo.com to see if there is a meta redirect?
From what I gather, when a site is hijacked then google will drop the site and PR0 it.
If that is the case then how quickly does that happen?
Is my site in the early stage of being dropped by google?
[google.com ]
Digital Millennium Copyright ActIt is our policy to respond to notices of alleged infringement that comply with the Digital Millennium Copyright Act (the text of which can be found at the U.S. Copyright Office Web Site, [lcWeb.loc.gov ]) and other applicable intellectual property laws, which may include removing or disabling access to material claimed to be the subject of infringing activity. If we remove or disable access to comply with the Digital Millennium Copyright Act, we will make a good-faith attempt to contact the owner or administrator of each affected site so that they may make a counter notification pursuant to sections 512(g)(2) and (3) of that Act. It is our policy to document all notices of alleged infringement on which we act. A copy of the notice will be sent to a third party who will make it available to the public.
Etc...
Google does not have any kind of duty or obligation to index anybody's website. If they decide they only want to show results for domains that begin with the letter 'A' and were registered on a Tuesday, that's their right. If you don't like how they operate, make your own search engine.
That said, I think this is a pretty major flaw in their algorithm and I, for one, would like to see more extensive analyses of the problem and how it could be resolved. And I'd really be interested in seeing something from GoogleGuy explaining why they do things this way. Enough whining about lawsuits, let's get back to discussion of the actual problem itself.
now, vista is sold. thats pity, yeah? :)
lynx -mime_header HIJACKURL
where HIJACKURL is the url of the potential hijacker, including "http..." Lynx will show you the HTTP headers, so you can tell if it's a 301, 302, or meta refresh.
Ahh, the good old days.
TO Frank_Rizzo:
Search Google for "wannabrowser".
under "Agent Selection" use "NetSpider"
UN-Check "Follow Redirects"
Leave "Show HTTP Response Headers" Checked
Yesterday I have send a DMCA-complaint to Google and to Altavista by Fax concerning this matter ... hopefully I will get an answer to this fax.
Today all my more than 8,000 backlinks to my site "www.widget.com" have disappeared.
entering link:www.widget.com now gives as answer:
"Your search - link:www.widget.com - did not match any documents."
So now the hijacking-page must have been completely accepted as "THE PAGE" and as Frank_Rizzo says in message 81, the next step will be that my PR6 will become PR0, resulting in a complete loss of:
- a 4-year old site
- over 80,000 pages
- PR6 ranking
- over 30,000 uniques/day
- 200,000 pageviews/day
All of the above the result of someone adding the following line of code to a not so high-ranking page:
"<meta http-equiv="refresh" content="0; url=http://www.widget.com/">"
Also Today (its morning here) My pages in the Google-Index have now dropped from over 80,000 pages to less than 40,000 pages. (using site:www.widget.com)
Also Google traffic is 50% less than the normal average from the last 6-months
I am watching Yahoo like a hawk as I am still getting a lot of traffic from them, but the hijacking-page is STILL NOT not in the Yahoo-Index and my www.widget.com page is still ranking No.1 for its main topic (keywords) on over 3-million results returned.
I still believe in Google and agree with "DaveAtIFG message 79" .... I just hope Google knows about the problem so that other webmasters never have this scenario happen to them.
