Welcome to WebmasterWorld Guest from 54.167.253.186

Forum Moderators: open

Message Too Old, No Replies

Dupe content checker - 302's - Page Jacking - Meta Refreshes

You make the call.

     
11:35 am on Sep 7, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:42
votes: 0


My site, lets call it: www.widget.com, has been in Google for over 5-years, steadily growing year by year to about 85,000 pages including forums and articles achieved, with a PageRank of 6 and 8287 backlinks in Google, No spam, No funny stuff, No special SEO techniques nothing.

Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.

At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.

The index-page was always in the top-3 position for our main topics, aka keywords.

I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index

As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.

My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.

Lets call them:
www.foo1.net and www.foo2.net

Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)

The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window

Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).

Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.

So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.

Fetched the page, but again got a code 200 with no redirects at all.

Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"

Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.

Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html

Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.

None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo

I have send an email to Google about this and am still waiting for a responds.

3:33 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2002
posts:894
votes: 0


More Info...

I did a header check on the 302 and got the following. Notice that there is a reference in the 302 for www.theirsite.com to www.anothersite.com.


#1 Server Response: [theirsite.com...]
HTTP Status Code: HTTP/1.1 302 Found
Date: Fri, 10 Sep 2004 15:07:50 GMT
Server: Apache/1.3.26 (Unix) PHP/4.3.0 FrontPage/5.0.2.2510
X-Powered-By: PHP/4.3.0
P3P: CP="CAO DSP COR CURa ADMa DEVa OUR IND PHY ONL UNI COM NAV INT DEM PRE" policyref="www.anothersite.com/w3c/p3p.xml"
Location: index.php
Connection: close
Content-Type: text/html
Redirect Target: index.php

I did a header check on www.anothersite.com/w3c/p3p.xml and got...

#1 Server Response: [www.anothersite.com...]
HTTP Status Code: HTTP/1.1 302 Found
Date: Fri, 10 Sep 2004 15:08:30 GMT
Server: Apache/1.3.27 (Unix) (Red-Hat/Linux) PHP/4.3.8
X-Powered-By: PHP/4.3.8
Location: /
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Redirect Target: /

This redirected to the www.anothersite.com main pages and guess what? It is a search optimization company with a very large directory of links split up in Yahoo type fashion complete with search capability and everything.

I randomly checked some of these links and lo and behold, all the links are meta refreshes to the sites listed. Must be several thousands of links here (maybe more, it's hard to tell). All the links on the pages are programmed so on a mouseover, it shows the correct URL in the bottom of the browser (just like normal links would), but the actual links are a php script that goes to a meta refresh page.

Looks like thousands of sites have gotten sucked into this thing.

5:30 pm on Sept 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:42
votes: 0


To Kales:
No the site is not cloaking, but I think they have added the IP of my (small) country to there htaccess file.
I myself can not load the site in my browser but my friends from abroad can.
The most strange is that I can spider the page with LWP but not with a normal browser.
#########
To Macro:
its not a "throwaway domain" they even run a adsense code on the page "www.foo.net/some-keyword/www-widget-com.html"
and they are premium as the adsense code is this of a primium member
#########
Again To Macro:
"And all you need is a throwaway domain with higher PR than theirs"
No, my page has a PR6 and has been hijacked by a PR3 page
My PR6 index-page (www.widget.com) has completely disappeared out of the Google index and been replaced by a PR3 page named "www.foo1.com/mykeyword/www-widget-com.html" which only contains the Meta-Refresh Tag and a Premium Adsense Code.
########
To Critter:
"assigning a"priority date to pages"
Thats where I think the Google-Bug is, as soon as I change the content of my page, my page becomes the newer page and the hijacker becomes the older page ... so for the duplicate content filter the spammer becomes the oldest page on the net and the updated original page is deleted from the Google-Index (only my theory ... no proove)
########
To Webdude"
"Google tells me not to use meta refreshes"
I think the Googlebot has a bug that can not see a content-refresh of 0-seconds
#########
To Momsbudget:
"would regard the older source as authoritative "
As soon as you update your content on your page, you get a new date, so that the spammer becomes the older source and the original source becomes the younger-source.
#########
To Beren:
"FBI and other law enforcement agencies interested"
What about first Google accepts some feedback an reads (and answers) the emails we are sending them.
Just fix the Google-bot and algo for those matters, their is enough feedback since months about this simple hijacking problem on webmasterworld
5:37 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 31, 2003
posts:2280
votes: 0


No, my page has a PR6 and has been hijacked by a PR3 page

That small comment could have major repercussions. So far it seemed that the meta problem site needed to have a higher PR to achieve this effect. If any PR3 site can take down a big boy are we saying that any joker can remove a PR10 website like .... oh, OK, no names?

5:41 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2002
posts:894
votes: 0


Hey Marcello,
Thats where I think the Google-Bug is, as soon as I change the content of my page, my page becomes the newer page and the hijacker becomes the older page ... so for the duplicate content filter the spammer becomes the oldest page on the net and the updated original page is deleted from the Google-Index (only my theory ... no proove)

Come to think about it, I made some major changes to all my pages right before this happened. I was waiting for a crawl to see what the results would be when the site was hijacked.

Funny, I didn't check the #50 link in the SERPs for a few days assuming it was my link. I mean, I would search my key phrase, see the link at #50 and think to myself, "Well, I am still at the same spot I was yesterday." Think of the surprise I got when I rolled over the link and noticed the link going to notmysite.com! And now my page GONE!

5:53 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2002
posts:894
votes: 0


The plot thickens! In fact, I would appreciate a sticky from an admin or mod who can tell me what this means. I think I am getting in WAY over my head and I need some very knowledgable people to help me figure out whether or not what I am finding is true.

When I go to the SEO company that has the meta refreshes (the thousands of links) and run the php link through an html checker to get the meta refresh, I am getting a refresh that may be triggering click throughs to a major search engine company. One that might even be listed as a forum on this site.

I need verification of this and whether I am on to something.

I don't want to jeopordize my company or any of my sites, so I need someone with knowledge of word bids, click throughs, etc.

Please, only mods, admins or seniors.

This is getting me scared.

6:05 pm on Sept 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:42
votes: 0



"I would appreciate a sticky from an admin or mod"

And I would appreciate a reaction from Google or an answer to the emails I have send and to Google and to Adsense

Ok its Midnight here ... i am of to bed

9:00 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member kaled is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 2, 2003
posts:3710
votes: 0


As I have said in other threads, Google need to be shamed into fixing this - that requires bad publicity.

How about this
1) Take a snaphot of a Google-cached page crediting the copyright to the wrong site.
2) Take a snapshot of the meta redirect code (by disabling such redirects in your browser).
3) Hire a lawyer.
4) Slap Google and the cheating scumbag site jointly with a 100 billion dollar lawsuit - that would probably be the largest claim for punitive damages in history. If that doesn't attract some publicity, we can only assume Google have secretly bought the world's news media.

On the matter of which of two or more pages is original, the only valid test is which page was first indexed. However, if Google don't keep that information then it isn't possible. That being the case, duplicates should only be removed if they are on the same site.

Kaled.

9:22 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 25, 2002
posts:776
votes: 0


shamed into fixing

This is a good point. Now that I fully understand the issues, I have got to say I'm outraged. None of my sites have been hijacked, yet, but this is something I should not have to worry about. As the news spreads, it is going to become more and more common.

Fixing this should be priority one for Google.

I recommend:
1) folks going to pubcon getting straight answers out of Google reps.

2) Bay area folks picketing the google-plex (I'm willing to join in). I'm thinking if we can get twenty or so of us down there with signs for half a day and alert the news, this problem will be fixed within a few days.

9:33 pm on Sept 10, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 31, 2003
posts:386
votes: 0


<mob>
YAAA!

Let's take them out back the woodshed and beat the sh*t out of them!
</mob>

:)

9:43 pm on Sept 10, 2004 (gmt 0)

Full Member

10+ Year Member

joined:Dec 20, 2003
posts:268
votes: 0


"run the php link through an html checker to get the meta refresh, I am getting a refresh that may be triggering click throughs to a major search engine company"

I think this aspect frustrates me the most... because the fix to change how to handle 301 and 302 redirects is seemingly simple there *has* to be some reason why it can't just be done simply. I have seen a good number of page hijackers are selling clicks. The search companies have branched off into marketing as a primary focus it seems and maybe this is one of the results.

9:56 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2003
posts:1246
votes: 0


very scary. i think i will burn a copy of my site to disc along with a snapshot of the google cache of my homepage, mail it to myself (to get a postmark date), and then NEVER open it until the day i need legal counsel to review it.

also, was wondering if periodically varying the wording of the index page's title might help.

10:17 pm on Sept 10, 2004 (gmt 0)

New User

10+ Year Member

joined:Sept 10, 2004
posts:5
votes: 0


Make sure it is certified mail. The postmark alone is not enough.
10:22 pm on Sept 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 13, 2003
posts:1233
votes: 1


It seems that the same search optimization company is continually being named as the culprit.I know dogboy was talking about these guys.
Why don't you get a few webmasters that are affected together and launch some legal action against them and file a dmca report to google.They are obliged to respond to these.
Send emails to Daniel Brandt at #*$! and try to get some press releases.
Don't sit and do nothing and hope it fixes itself.Get heavy!
and good luck!
11:26 pm on Sept 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 3, 2003
posts:143
votes: 0


You could spider the offending sites directory for email addresses and send an email to all of the webmasters of the sites being affected.

I'm sure a class action lawyer wouldn't mind getting some press for this.

This is very disturbing, especially if monitary gains are being made by the people doing this.

Do no evil? huh.... sounds pretty evil to me.

11:27 pm on Sept 10, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:May 4, 2004
posts:642
votes: 0


Well if Google won't fix it, most people will take advantage of it and I won't blame them.
This 389 message thread spans 26 pages: 389