Marcello
msg:59536 | 9:20 am on Sep 10, 2004 (gmt 0) |
After further investigation I have now found out that foo.net hijacks good-performing pages with a simple "http-equiv=refresh" Meta-Tag. Using LWP here is how the HTML-head of their pages looks like:
<html>
<head>
<title>Copied Title of the Hijacked Page</title>
<meta http-equiv="refresh" content="0; url=http://www.widget.com/">
<meta name="robots" content="follow, noindex">
</head>
<body bgcolor="#FFFFFF" text="#000000">
..... Further HTML Code ..... I simply can not understand why Google replaces good-performing index-pages like my:
"www.widget.com"
by a Meta-Refresh (redirect) page as:
"www.foo.net/some-keyword/www-widget-com.html" This is really a huge BUG in Google's algo
|
kaled
msg:59537 | 11:27 am on Sep 10, 2004 (gmt 0) |
This subject has been discussed at depth but you may have discovered a new variation. [webmasterworld.com...] There was also a more recent thread but I don't have a link to it. It sounds as though the site may be cloaking so that it only delivers content to Googlebot and refuses browsers - either that or the site is offline. Try the following link. You may try setting the user agent to googlebot to see what response you get. Alternatively, using Firefox¦about:config you can change the user agent strings I think - never tried it.
http://www.rexswain.com/httpview.html Hope this helps Kaled.
|
Macro
msg:59538 | 11:37 am on Sep 10, 2004 (gmt 0) |
This sort of activity has got to be criminal. Why isn't it? Any webmaster who's devoted considerable time and effort (not to say expense) on his site is vulnerable and the perpetrators pay no cost. Their throwaway domain will be, er, thrown away when they are caught and the only loser is the innocent party.
|
DaveN
msg:59539 | 11:38 am on Sep 10, 2004 (gmt 0) |
This is really a huge BUG in Google's algo .... simple answer is YES it is.... DaveN
|
Macro
msg:59540 | 11:42 am on Sep 10, 2004 (gmt 0) |
| simple answer is YES it is. |
|
And they've apparently known about it for ages (going by other posts and webmasters who've complained). Why haven't they done anything about it? We know their penchance for automated solutions over hand editing but considering that it's "BIG" problem surely they'd have found an automated solution with some alacrity?
|
webdude
msg:59541 | 12:07 pm on Sep 10, 2004 (gmt 0) |
The other thread... [webmasterworld.com...] And after searching I found many threads, all with different variations of the same problem. [webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...] Many more that were kind of related. And lots of stuff on the Yahoo forum too. So what does it take to fix this? I could very easily do this to all my competition, not to mention all my outgoing links.
|
Macro
msg:59542 | 12:10 pm on Sep 10, 2004 (gmt 0) |
| I could very easily do this to all my competition |
|
Apparently so. And all you need is a throwaway domain with higher PR than theirs. No html, php or hacking skills required. Simply cut and paste a meta tag from the examples and replace the URL in it. I just went through all those same threads that you found and it seems the problem dates back to last year (or, at least, Jan this year). Google, how come you are so silent on this?
|
Critter
msg:59543 | 1:34 pm on Sep 10, 2004 (gmt 0) |
Google could probably prevent this sort of thing by assigning a "priority date" to pages it crawls. If googlebot then finds duplicate content on the net it would regard the older source as authoritative and remove the *newer* duplicate content. Google's engineers may have been drinking decaf at that meeting. :)
|
Macro
msg:59544 | 1:47 pm on Sep 10, 2004 (gmt 0) |
| If googlebot then finds duplicate content on the net it would regard the older source as authoritative |
|
That's what I thought they did till I saw one of my internal pages copied verbatim and the copied page ranking higher than ours in SERPS (the copy does have a higher PR). <sigh> Just sent them a cease and desist.
|
webdude
msg:59545 | 2:07 pm on Sep 10, 2004 (gmt 0) |
Everything I have studied and read on meta refreshes and Google tells me not to use meta refreshes. It is bad practice and could be subject to removal by Google. I always thought that was the case, so I never used them. I can find hundreds of documents supporting this. All of them say that if neccessary, set the refresh rate high or you will be penalized by Google. Just search "google" "meta refresh" and you will see what I mean. Also, almost all references to meta refresh in google answers say that it is a bad practice to use them. In fact, it was considered a popular spam ploy years ago and was used frequently by spam sites. Something changed and changed drastically. Now there are many meta refreshes showing up in the SERPs. And if I got the timing right, this started right around the Florida fiasco. I read all these posts of webmasters losing pages in the SERPs, sudden drops in referrals, etc etc., especially since Florida and I wonder if some of this might be related. I have found several of my sites with this problem. Thankfully, the other sites rank higher then the refreshes, and have stayed that way for some time. But the site now in question has been REPLACED by a meta refresh. Two weeks ago, my home page ranked #50 for a specific key phrase, as of last Monday, the #50 spot is a link to a different site with a meta refresh to my site. My page is gone. Pretty neat trick if you ask me. I think it needs to be fixed. Google is now broken and anyone can do this. I wonder how many sites will fall before the G takes notice?
|
webdude
msg:59546 | 2:42 pm on Sep 10, 2004 (gmt 0) |
By the way, I just noticed that the link on Google that goes to the meta refresh page is now #5. That should be my site there. The link has been changed to a 302 redirect to this sites home page. I think they are doing a double hop now. A meta refresh to a 302 redirect. I need to investigate some more. :-( A confused and totally bummed out WebDude p.s. My home page is still gone for all key phrases. It is still in the index though if I search the title. My home page is #1, a sub page of mine is #2. Guess what is #3? You got it! The site with the meta refresh that goes to their home page! And if I do an exact search on the title, my site is #1 and the site with the meta refresh is #2! ARGGGGGGGGGGGGGG!
|
GranPops
msg:59547 | 2:59 pm on Sep 10, 2004 (gmt 0) |
Happened with dramatic effect on one of my sites.
Dropped out of sight, got PR0, and his cache was my page. Fortunately it was an error on the part of the person who coded his links prog. On removing all his links the problem went away. Unfortunately the process took so long, with zero interest on the part of G, that I had abandoned the domain. On another site which had one page duplicated, my page is now back in the cache, and PR has returned.
|
momsbudget
msg:59548 | 3:04 pm on Sep 10, 2004 (gmt 0) |
<If googlebot then finds duplicate content on the net it would regard the older source as authoritative >
<That's what I thought they did till I saw one of my internal pages copied verbatim and the copied page ranking higher than ours in SERPS (the copy does have a higher PR). <sigh> Just sent them a cease and desist.> That's what I thought as well, but clearly it's not as I've had to send out over 20 C&D's in the past two days for this exact event.
|
beren
msg:59549 | 3:11 pm on Sep 10, 2004 (gmt 0) |
| This sort of activity has got to be criminal. Why isn't it? |
|
I'm sure it is. We need to get the FBI and other law enforcement agencies interested in this crime.
|
webdude
msg:59550 | 3:33 pm on Sep 10, 2004 (gmt 0) |
More Info... I did a header check on the 302 and got the following. Notice that there is a reference in the 302 for www.theirsite.com to www.anothersite.com.
#1 Server Response: [theirsite.com...]
HTTP Status Code: HTTP/1.1 302 Found
Date: Fri, 10 Sep 2004 15:07:50 GMT
Server: Apache/1.3.26 (Unix) PHP/4.3.0 FrontPage/5.0.2.2510
X-Powered-By: PHP/4.3.0
P3P: CP="CAO DSP COR CURa ADMa DEVa OUR IND PHY ONL UNI COM NAV INT DEM PRE" policyref="www.anothersite.com/w3c/p3p.xml"
Location: index.php
Connection: close
Content-Type: text/html
Redirect Target: index.php
|
|
I did a header check on www.anothersite.com/w3c/p3p.xml and got...
#1 Server Response: [www.anothersite.com...]
HTTP Status Code: HTTP/1.1 302 Found
Date: Fri, 10 Sep 2004 15:08:30 GMT
Server: Apache/1.3.27 (Unix) (Red-Hat/Linux) PHP/4.3.8
X-Powered-By: PHP/4.3.8
Location: /
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Redirect Target: /
|
|
This redirected to the www.anothersite.com main pages and guess what? It is a search optimization company with a very large directory of links split up in Yahoo type fashion complete with search capability and everything. I randomly checked some of these links and lo and behold, all the links are meta refreshes to the sites listed. Must be several thousands of links here (maybe more, it's hard to tell). All the links on the pages are programmed so on a mouseover, it shows the correct URL in the bottom of the browser (just like normal links would), but the actual links are a php script that goes to a meta refresh page. Looks like thousands of sites have gotten sucked into this thing.
|
Marcello
msg:59551 | 5:30 pm on Sep 10, 2004 (gmt 0) |
To Kales:
No the site is not cloaking, but I think they have added the IP of my (small) country to there htaccess file.
I myself can not load the site in my browser but my friends from abroad can.
The most strange is that I can spider the page with LWP but not with a normal browser.
#########
To Macro:
its not a "throwaway domain" they even run a adsense code on the page "www.foo.net/some-keyword/www-widget-com.html"
and they are premium as the adsense code is this of a primium member
#########
Again To Macro:
"And all you need is a throwaway domain with higher PR than theirs"
No, my page has a PR6 and has been hijacked by a PR3 page
My PR6 index-page (www.widget.com) has completely disappeared out of the Google index and been replaced by a PR3 page named "www.foo1.com/mykeyword/www-widget-com.html" which only contains the Meta-Refresh Tag and a Premium Adsense Code.
########
To Critter:
"assigning a"priority date to pages"
Thats where I think the Google-Bug is, as soon as I change the content of my page, my page becomes the newer page and the hijacker becomes the older page ... so for the duplicate content filter the spammer becomes the oldest page on the net and the updated original page is deleted from the Google-Index (only my theory ... no proove)
########
To Webdude"
"Google tells me not to use meta refreshes"
I think the Googlebot has a bug that can not see a content-refresh of 0-seconds
#########
To Momsbudget:
"would regard the older source as authoritative "
As soon as you update your content on your page, you get a new date, so that the spammer becomes the older source and the original source becomes the younger-source.
#########
To Beren:
"FBI and other law enforcement agencies interested"
What about first Google accepts some feedback an reads (and answers) the emails we are sending them.
Just fix the Google-bot and algo for those matters, their is enough feedback since months about this simple hijacking problem on webmasterworld
|
Macro
msg:59552 | 5:37 pm on Sep 10, 2004 (gmt 0) |
| No, my page has a PR6 and has been hijacked by a PR3 page |
|
That small comment could have major repercussions. So far it seemed that the meta problem site needed to have a higher PR to achieve this effect. If any PR3 site can take down a big boy are we saying that any joker can remove a PR10 website like .... oh, OK, no names?
|
webdude
msg:59553 | 5:41 pm on Sep 10, 2004 (gmt 0) |
Hey Marcello,
| Thats where I think the Google-Bug is, as soon as I change the content of my page, my page becomes the newer page and the hijacker becomes the older page ... so for the duplicate content filter the spammer becomes the oldest page on the net and the updated original page is deleted from the Google-Index (only my theory ... no proove) |
|
Come to think about it, I made some major changes to all my pages right before this happened. I was waiting for a crawl to see what the results would be when the site was hijacked. Funny, I didn't check the #50 link in the SERPs for a few days assuming it was my link. I mean, I would search my key phrase, see the link at #50 and think to myself, "Well, I am still at the same spot I was yesterday." Think of the surprise I got when I rolled over the link and noticed the link going to notmysite.com! And now my page GONE!
|
webdude
msg:59554 | 5:53 pm on Sep 10, 2004 (gmt 0) |
The plot thickens! In fact, I would appreciate a sticky from an admin or mod who can tell me what this means. I think I am getting in WAY over my head and I need some very knowledgable people to help me figure out whether or not what I am finding is true. When I go to the SEO company that has the meta refreshes (the thousands of links) and run the php link through an html checker to get the meta refresh, I am getting a refresh that may be triggering click throughs to a major search engine company. One that might even be listed as a forum on this site. I need verification of this and whether I am on to something. I don't want to jeopordize my company or any of my sites, so I need someone with knowledge of word bids, click throughs, etc. Please, only mods, admins or seniors. This is getting me scared.
|
Marcello
msg:59555 | 6:05 pm on Sep 10, 2004 (gmt 0) |
"I would appreciate a sticky from an admin or mod" And I would appreciate a reaction from Google or an answer to the emails I have send and to Google and to Adsense Ok its Midnight here ... i am of to bed
|
kaled
msg:59556 | 9:00 pm on Sep 10, 2004 (gmt 0) |
As I have said in other threads, Google need to be shamed into fixing this - that requires bad publicity. How about this
1) Take a snaphot of a Google-cached page crediting the copyright to the wrong site.
2) Take a snapshot of the meta redirect code (by disabling such redirects in your browser).
3) Hire a lawyer.
4) Slap Google and the cheating scumbag site jointly with a 100 billion dollar lawsuit - that would probably be the largest claim for punitive damages in history. If that doesn't attract some publicity, we can only assume Google have secretly bought the world's news media. On the matter of which of two or more pages is original, the only valid test is which page was first indexed. However, if Google don't keep that information then it isn't possible. That being the case, duplicates should only be removed if they are on the same site. Kaled.
|
figment88
msg:59557 | 9:22 pm on Sep 10, 2004 (gmt 0) |
This is a good point. Now that I fully understand the issues, I have got to say I'm outraged. None of my sites have been hijacked, yet, but this is something I should not have to worry about. As the news spreads, it is going to become more and more common. Fixing this should be priority one for Google. I recommend:
1) folks going to pubcon getting straight answers out of Google reps. 2) Bay area folks picketing the google-plex (I'm willing to join in). I'm thinking if we can get twenty or so of us down there with signs for half a day and alert the news, this problem will be fixed within a few days.
|
Critter
msg:59558 | 9:33 pm on Sep 10, 2004 (gmt 0) |
<mob>
YAAA! Let's take them out back the woodshed and beat the sh*t out of them!
</mob> :)
|
idoc
msg:59559 | 9:43 pm on Sep 10, 2004 (gmt 0) |
"run the php link through an html checker to get the meta refresh, I am getting a refresh that may be triggering click throughs to a major search engine company" I think this aspect frustrates me the most... because the fix to change how to handle 301 and 302 redirects is seemingly simple there *has* to be some reason why it can't just be done simply. I have seen a good number of page hijackers are selling clicks. The search companies have branched off into marketing as a primary focus it seems and maybe this is one of the results.
|
ownerrim
msg:59560 | 9:56 pm on Sep 10, 2004 (gmt 0) |
very scary. i think i will burn a copy of my site to disc along with a snapshot of the google cache of my homepage, mail it to myself (to get a postmark date), and then NEVER open it until the day i need legal counsel to review it. also, was wondering if periodically varying the wording of the index page's title might help.
|
mccleark
msg:59561 | 10:17 pm on Sep 10, 2004 (gmt 0) |
Make sure it is certified mail. The postmark alone is not enough.
|
cabbie
msg:59562 | 10:22 pm on Sep 10, 2004 (gmt 0) |
It seems that the same search optimization company is continually being named as the culprit.I know dogboy was talking about these guys.
Why don't you get a few webmasters that are affected together and launch some legal action against them and file a dmca report to google.They are obliged to respond to these.
Send emails to Daniel Brandt at #*$! and try to get some press releases.
Don't sit and do nothing and hope it fixes itself.Get heavy!
and good luck!
|
my3cents
msg:59563 | 11:26 pm on Sep 10, 2004 (gmt 0) |
You could spider the offending sites directory for email addresses and send an email to all of the webmasters of the sites being affected. I'm sure a class action lawyer wouldn't mind getting some press for this. This is very disturbing, especially if monitary gains are being made by the people doing this. Do no evil? huh.... sounds pretty evil to me.
|
bears5122
msg:59564 | 11:27 pm on Sep 10, 2004 (gmt 0) |
Well if Google won't fix it, most people will take advantage of it and I won't blame them.
|
| This 389 message thread spans 13 pages: 389 ( [1] 2 3 4 5 6 7 8 9 ... 13 ) > > |
|
|
