homepage Welcome to WebmasterWorld Guest from 174.129.103.100
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 389 message thread spans 13 pages: < < 389 ( 1 [2] 3 4 5 6 7 8 9 10 ... 13 > >     
Dupe content checker - 302's - Page Jacking - Meta Refreshes
You make the call.
Marcello




msg:59535
 11:35 am on Sep 7, 2004 (gmt 0)

My site, lets call it: www.widget.com, has been in Google for over 5-years, steadily growing year by year to about 85,000 pages including forums and articles achieved, with a PageRank of 6 and 8287 backlinks in Google, No spam, No funny stuff, No special SEO techniques nothing.

Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.

At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.

The index-page was always in the top-3 position for our main topics, aka keywords.

I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index

As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.

My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.

Lets call them:
www.foo1.net and www.foo2.net

Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)

The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window

Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).

Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.

So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.

Fetched the page, but again got a code 200 with no redirects at all.

Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"

Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.

Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html

Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.

None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo

I have send an email to Google about this and am still waiting for a responds.

 

mivox




msg:59565
 11:32 pm on Sep 10, 2004 (gmt 0)

mail it to myself (to get a postmark date)

Be extra thorough: take the sealed envelope to a notary public, have them stamp and sign the envelope OVER the sealed flap, put a piece of glass-tape over the notary seal, and THEN have it sent to yourself via certified mail.

Or, better yet, register the contents of the CD with the copyright office. ;)

ownerrim




msg:59566
 1:37 am on Sep 11, 2004 (gmt 0)

I wish I understood these issues better but unfortunately I don't. The hijacker duplicates a page and then inserts a meta refresh tag to redirect back to the original page, right?

So, if this is the technique for hijacking, what is the actual trigger that causes google to think the hijacker's page is the real one? Is it the fact that the hijacker's page is identical? If so, couldn't you beat this by periodically varying the content of your own page (say, by altering body text slightly or changing page titles slightly)?

And this next statement I don't follow:

"Thats where I think the Google-Bug is, as soon as I change the content of my page, my page becomes the newer page and the hijacker becomes the older page ... so for the duplicate content filter the spammer becomes the oldest page on the net and the updated original page is deleted from the Google-Index"

What I don't follow is this: if he changes the content of his page, how can a duplicate content filter apply? They're no longer identical copies and, logically, there should be no duplicate content penalty?

But, back to the first question, what is it specifically that causes google to think the bogus page is the real one?

markus007




msg:59567
 2:45 am on Sep 11, 2004 (gmt 0)

I don't see how you could sue google or anyone else for that matter. There is no contract between your site and google therefor you have no grounds to sue.

jdMorgan




msg:59568
 5:25 am on Sep 11, 2004 (gmt 0)

> So, if this is the technique for hijacking, what is the actual trigger that causes google to think the hijacker's page is the real one? Is it the fact that the hijacker's page is identical?

The trigger is that Google is treating a meta-refresh like a 302-Found (or Moved Temporarily) redirect. As described by the HTTP/1.x specifications, the effect of a 302 is to cause Googlebot to index the URL of the page containing the 302 with the content of the page the 302 redirects to.

So, changing your page content won't help, because it is your own content that is displayed with their URL.

This is the result of Google treating meta-refresh as a 302, rather than as a 301. With a 301, the new URL is indexed along with its content, so there is no problem.

I posted this "thinking out loud" in another thread, but let me try again: If anyone has had this happen to them, what is the effect if you add a 301 redirect to the end of this redirection chain? In other words, replace your hijacked page with a 301-Moved Permanently redirect to a copy of your page. I'm wondering if this final redirect might nullify the meta-refresh being treated as a 302, and cause Googlebot to decide that the final page URL should be listed with its content. There are risks, or course: It might be a good idea to only present the 301 to Googlebot, and not to other SE spiders if your ranking is OK in those other SE's listings. However, if someone is suffering from this problem, and has nothing to lose by trying it, this would be an interesting experiment.

I hope Google will fix the root problem, but this might be a work-around if Googlebot gives precedence to the last redirect they encounter.

Jim

AprilS




msg:59569
 5:51 am on Sep 11, 2004 (gmt 0)

ownerrim -
the offending site doesn't need to actually "copy" the content to their site. The Meta Refresh redirect is enough. This type of redirect tells Google that the content has temporarily moved. Doing so, Google then stores in its index exactly what you have on your page in the offending sites index record.

markus007 -
I'm not sure either...as I'm not a lawyer. However, I'm sure there are actions you can take towards a company that knowingly allows other sites to do something that "knowingly" can hurt someones site resulting in a loss in revenue. This issue is not new... but within the past few weeks it has just BALLOONED on webmasterworld. I really do not think Google can say they had now idea.

And for the record, my site as well is a PR6 and I was brought to my knees by a lower PR site.

I don't think it is too much to ask for Google to respond. If they can't fix their algo. immediately... then they could AT LEAST provide a report page SPECIFICALLY for this issue and actually remove/penalize the offending sites/pages from the index! Is that really too much to ask?

This is one of the largest issues I've seen with Google. Yes, algorithm changes suck because it requires people to adjust their sites. But when something comes along that can destroy online companies that have been around for YEARS with high PR..etc and Google (& other search engines) not take immediate action - That is WRONG!

shri




msg:59570
 6:10 am on Sep 11, 2004 (gmt 0)

As an FYI, a couple of well known links engines default to 302 redirects in the "jump.cgi" and php scripts.

Marcello




msg:59571
 8:26 am on Sep 11, 2004 (gmt 0)

To jdMorgan:
Thanks for the concern.
Could you eventually post an example how to add this final 301 redirect in a HTaccess file, and only presenting this to Google

presuming my site is "www.example.com" and the hijacked page is:
[example.com...]
and/or
[example.com...]

To All Members:
1/ Can someone post any kind of Channel or e-maill address to point Google-Staff to this threat or where I can reach them.
e-mails to "webmaster at google.com" and "help at google.com" send last week are still unanswered.

2/ I am receiving tons of Sticky-Mails to send the URL of the hijacking-site, mails from posters in this threat but also many from non-posters.

I currently will not respond by sending the URL as I think there is some danger that, excuse me for this, would-be hijackers want to see the complete code this URL is using, being that the Google-bot and algo is currently so easely fooled by such a simple trick.

I still believe in Google and hope that in one or other way they will gey notified, fix the problem and close the door for all future uses of this trick.

[edited by: DaveAtIFG at 6:46 pm (utc) on Sep. 11, 2004]
[edit reason] Exemplified URLs [/edit]

kaled




msg:59572
 9:35 am on Sep 11, 2004 (gmt 0)

There is no contract between your site and google therefor you have no grounds to sue

There is no need for a contract to exist. If your neighbor were to fill a giant water-gun with weed-killer and spray your lawn with it, you could sue. Strangely, neighbors are not required to sign contracts with each other.

Suing Google would force them to fix the problem. However, it would be expensive.

I think it's worth noting that at least one webmaster here has taken legal action with respect to copyright.

SUGGESTION
If you have no understanding of law, refrain from quoting it.

Kaled.

GranPops




msg:59573
 10:37 am on Sep 11, 2004 (gmt 0)

AprilS
"the offending site doesn't need to actually "copy" the content to their site. The Meta Refresh redirect is enough. This type of redirect tells Google that the content has temporarily moved. Doing so, Google then stores in its index exactly what you have on your page in the offending sites index record"

Precisely what happened to our sites.

Kaled.
If we agree to your suggestion not to disseminate MSB, the forum would be virtually empty. LOL

Larryhat




msg:59574
 10:51 am on Sep 11, 2004 (gmt 0)

Hello all:

Forgive a dumb question, I'm pretty dense when it
comes to redirects etc.

Q: What is the simplest way to determine if another site is hijacking my content as described in this thread?

I have one possible scumbag in mind.

Thanks in advance. - Larry

Kerrin




msg:59575
 12:30 pm on Sep 11, 2004 (gmt 0)

If anyone has had this happen to them, what is the effect if you add a 301 redirect to the end of this redirection chain? In other words, replace your hijacked page with a 301-Moved Permanently redirect to a copy of your page.

Geocities page =[meta refresh]> NoLongerActive.widget.com =[301]> www.widget.com

The above resulted in the Geocities page showing www.widget.com's title, description, cache, backlinks and PR. The real www.widget.com page was removed from Google and the most of my site was dropped in the same way Marcello described in post #1. I ended up 404ing NoLongerActive.widget.com which at least got www.widget.com back in Google but not the pages that were dropped.

E-mails to Google about the bug and the resulting penalty got me:
- 2 canned responses saying "we don't comment on site penalties"
- 1 canned response saying "we'll pass this on to our engineers"
- No reply when I asked for an update a month later (July)

tombola




msg:59576
 1:45 pm on Sep 11, 2004 (gmt 0)

Apparently, till now, Google preferred not to do anything.

This means there are still two parties left that can stop this abuse:

1. The offender.
2. The judge.

Contact the offender with the explicit instruction to remove all these refresh meta tags immediately.
If you don't get a response within let's say 1 business day, then take legal actions.

... wondering how fast Google will change its policy when the first lawsuits will be filed ;-)

ownerrim




msg:59577
 2:11 pm on Sep 11, 2004 (gmt 0)

hmm. I have a site on a shared server and, in reviewing the stats, I see there's always been x amount of hits attributed to 302 redirects. I just assumed it had something to do with the hosting arrangement. Could that be it, or could it be something else?

Bluepixel




msg:59578
 2:24 pm on Sep 11, 2004 (gmt 0)

You have to blame google, not the site owner.

It's their fault. You are allowed to redirect to a site with a 301 redirect or a meta refresh. It's your site and not google's and you don't copy anything from the other site.

Patrick Taylor




msg:59579
 2:55 pm on Sep 11, 2004 (gmt 0)

the effect of a 302 is to cause Googlebot to index the URL of the page containing the 302 with the content of the page the 302 redirects to.

<?php
$location = 'ht*p://somesite.com/';
header('Location: ' . $location);
?>

The above returns a 302 also, from an url like ht*p://www.mysite.com/link.php?url=somepage. It is commonly (and innocently) used in php redirect scripts which sit in their own (normally unseen) file. I use it myself to help track outgoing clicks. Surely link.php isn't credited with the content of all the pages it links to?

Bluepixel




msg:59580
 3:44 pm on Sep 11, 2004 (gmt 0)

If the ranking for your link would be higher than the ranking of the link you link to, it would get replaced and your link.php?id=.... would get the content.

zomega42




msg:59581
 3:57 pm on Sep 11, 2004 (gmt 0)

Let's get the facts straight... Is this happening only with META refreshes, or also with regular 302's (like the php example)?

A client's homepage disappeared, I have been trying to figure out why. There is one site that shows up when searching "client's company name" on google, which is a 302 redirect script (.cgi file). Could this be causing the problem, or does it only apply to meta refresh?

Bluepixel




msg:59582
 4:10 pm on Sep 11, 2004 (gmt 0)

It happened with "Location:http://..." redirects on my site. I don't know if it still works, I removed my redirects, but it worked like 2 months before.

kaled




msg:59583
 5:54 pm on Sep 11, 2004 (gmt 0)

does it only apply to meta refresh?
In the absence of proof to the contrary, assume that redirect methods other than METAs will also cause this problem.

This problem was first described with META redirects, but scumbags being scumbags, they will have experimented to see what else works.

It is likely that the META redirect is grouped for behavioural purposes with others. It is likely that the fault lies in the behaviour of the group rather than special case code for METAs.

Kaled.

Jebus




msg:59584
 6:29 pm on Sep 11, 2004 (gmt 0)

Is this the same idea of what everyone is talking about in this thread? Theirsite.com links to mine but I dont know why they do it this way. Here is a copy of it:

----------------------------------------------------

<script language="Javascript" src="http://theirsite.com/clients/script.aspx?campaignid=2915&websiteid=2237&scriptid=11868&subwebsiteid="></script>
<meta http-equiv='Refresh' content='0; url=http://www.mysite.com/index.html'>

---------------------------------------------------

Both [theirsite.com...] and http://mysite.com are indexed and I can find both in searches, however, only when i click on "If you like, you can repeat the search with the omitted results included." I can find both.

So pretty much the two pages are identical in google.

[edited by: DaveAtIFG at 6:54 pm (utc) on Sep. 11, 2004]

Marcello




msg:59585
 6:48 pm on Sep 11, 2004 (gmt 0)

Yes Jebus -- it's the same

You have been hijacked with the simple:
<meta http-equiv='Refresh' content='0; url=http://www.mysite.com/index.html'> tag on the other sites page.

At one moment Google's duplicate content filter will see that both pages have the same content and your page will get booted out of the Serps - If your page has a good PageRank then this will also go to the other site.

internetheaven




msg:59586
 7:45 pm on Sep 11, 2004 (gmt 0)

The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window

Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).

Am I missing something? If the page cannot be accessed by regular users then what is the purpose of these pages - simply to get you kicked off?

AprilS




msg:59587
 8:01 pm on Sep 11, 2004 (gmt 0)

internetheaven -
Yes & No. I think there are definately people out there doing this on purpose. Not to get you "kicked out" - but to better their site rankings! No, does this mean all offending sites are doing it on purpose - I don't think so - as there are linking programs out there (I believe one called XOOPS) that use this linking strategy - and some people may be using it an not knowing the effect it could have on other sites.

Marcello




msg:59588
 10:02 pm on Sep 11, 2004 (gmt 0)

internetheaven - "Am I missing something"

I think you missed this tag on their page (see the second message in this threat):
<meta name="robots" content="follow, noindex">

Seems the meaning is that the page should not be found.
But Google is a little different than other engines as with "noindex" it still shows the URL and title in the serps but without text-snippets.

The normally not so important hijacking page has now taken over the PR6 from a page and their whole site's ranking is upgraded by this.

Do this with another 100 or 500 high PR-pages sites!

markus007




msg:59589
 1:24 am on Sep 12, 2004 (gmt 0)


SUGGESTION
If you have no understanding of law, refrain from quoting it.

Kaled.

The example you gave has absolutely nothing to do with this thread or situation. There is a BIG difference between owning something and someone giving you traffic for free and then stop giving it.

If some site where to be allowed to sue google then using the same logic i could sue any site that puts up a link to my site and then removes it for whatever reason.

Patrick Taylor




msg:59590
 2:19 am on Sep 12, 2004 (gmt 0)

There is a BIG difference between owning something and someone giving you traffic for free and then stop giving it.

Very true. The word "crime" has been used in this thread - wrongly, I believe. This is more about Google rankings than about stealing content.

1milehgh80210




msg:59591
 2:40 am on Sep 12, 2004 (gmt 0)

Where's all those people who spout 'if your business is dependant on free google traffic, your business plan is unsound'? lol

But seriously, G needs to fix this crack in their algo before it becomes a crater.

paully11




msg:59592
 3:04 am on Sep 12, 2004 (gmt 0)

Another person in this post already asked this, but didn't get a response. I would also like to know...

What is the easiest way to find out if your site is a victim of this type of activity?

My company's ecommerce site recently lost massive rankings on google, and I would like to find out if this could be the cause.

Thanks! Hope to see some of you at the upcoming conference in Vegas, baby, VEGAS!

Broadway




msg:59593
 3:36 am on Sep 12, 2004 (gmt 0)

I realize a number of people have already done this and I realize that Google is already aware of this problem but shouldn't everyone reading this thread still write Google at "webmaster@" and request that they address this issue? The squeeky wheel gets the oil.

kwngian




msg:59594
 7:05 am on Sep 12, 2004 (gmt 0)


It seems like google don't really ban any site nowsaday thus the increase in all these spamming tricks. Is this correct?

If google were to impose a ban on sites that have more than 50% redirects, will it work?

Bluepixel




msg:59595
 7:24 am on Sep 12, 2004 (gmt 0)

They should simply fix their software. They didn't do it before, and they should handle redirects on foreign domains as simple links.

This 389 message thread spans 13 pages: < < 389 ( 1 [2] 3 4 5 6 7 8 9 10 ... 13 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved