Welcome to WebmasterWorld Guest from 54.198.108.19

Forum Moderators: open

Message Too Old, No Replies

Dupe content checker - 302's - Page Jacking - Meta Refreshes

You make the call.

     
11:35 am on Sep 7, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:42
votes: 0


My site, lets call it: www.widget.com, has been in Google for over 5-years, steadily growing year by year to about 85,000 pages including forums and articles achieved, with a PageRank of 6 and 8287 backlinks in Google, No spam, No funny stuff, No special SEO techniques nothing.

Normally the site grows at a tempo of 200 to 500 pages a month indexed by Google and others ... but since about 1-week I noticed that my site was loosing about
5,000 to 10,000 pages a week in the Google Index.

At first I simply presumed that this was the unpredictable Google flux, until yesterday, the main index-page from www.widget.com disappeared completely our of the Google index.

The index-page was always in the top-3 position for our main topics, aka keywords.

I tried all the techniques to find my index page, such as: allinurl:, site:, direct link etc ... etc, but the index page has simply vanished from the Google index

As a last resource I took a special chunk of text, which can only belong to my index-page: "company name own name town postcode" (which is a sentence of 9
words), from my index page and searched for this in Google.

My index page did not show up, but instead 2 other pages from other sites showed up as having the this information on their page.

Lets call them:
www.foo1.net and www.foo2.net

Wanting to know what my "company text" was doing on those pages I clicked on:
www.foo1.com/mykeyword/www-widget-com.html
(with mykeyword being my site's main topic)

The page could not load and the message:
"The page cannot be displayed"
was displayed in my browser window

Still wanting to know what was going on, I clicked " Cached" on the Google serps ... AND YES ... there was my index-page as fresh as it could be, updated only yesterday by Google himself (I have a daily date on the page).

Thinking that foo was using a 301 or 302 redirect, I used the "Check Headers Tool" from
webmasterworld only to get a code 200 for my index-page on this other site.

So, foo is using a Meta-redirect ... very fast I made a little robot in perl using LWP and adding a little code that would recognized any kind of redirect.

Fetched the page, but again got a code 200 with no redirects at all.

Thinking the site of foo was up again I tried again to load the page and foo's page with IE, netscape and Opera but always got:
"The page cannot be displayed"

Tried it a couple of times with the same result: LWP can fetch the page but browsers can not load any of the pages from foo's site.

Wanting to know more I typed in Google:
"site:www.foo1.com"
to get a huge load of pages listed, all constructed in the same way, such as:
www.foo1.com/some-important-keyword/www-some-good-site-com.html

Also I found some more of my own best ranking pages in this list and after checking the Google index all of those pages from my site has disappeared from the Google index.

None of all the pages found using "site:www.foo1.com" can be loaded with a browser but they can all be fetched with LWP and all of those pages are cached in their original form in the Google-Cache under the Cache-Link of foo

I have send an email to Google about this and am still waiting for a responds.

4:34 pm on Sept 12, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 9, 2003
posts:359
votes: 0



Hello Patrick

Do you forsee the issue to be solved real soon?

The problem has been there for months already. There were no response or any assurance from google or the other search engines that the issue will be addressed.

The hijackers are thriving on it and may be enjoying the harvest through the coming holiday seasons. They are hoping everyone will be sitting on it and not do anything.

I know it is not the solution but is there any other way those affected can do instead of waiting? Can someone at google respond?

I have only afew pages affected thus not too concerned but I know that the scope that these hijackers hit at are quite broad.

5:09 pm on Sept 12, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member kaled is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 2, 2003
posts:3710
votes: 0


and as useless as trying to take Google to court over something that is most likely being worked on in any case

Taking Google to court is probably not the right action, however, this idea that the problem is being worked on is one I just don't get.

When a security flaw is found in Windows, MS can get a fix out in days (not always, but it does happen). This problem has been known for months, possibly more than a year and still no fix is in sight, nor have Google even conceded that the problem exists.

The possible explanations for this tardiness are as follows :-
1) Google just doesn't care.
2) The code is incomprehensible and the guy that wrote it has left/died, etc.
3) The code is fine but no one is smart enough to understand it.
4) The source code has been lost and no one is smart enough to work out how to patch the binary code.

Of course, the last possibility is interesting. They would probably need someone in his/her fifties or sixties to patch binary code - it's certainly not a skill taught at university.

Kaled.

6:13 pm on Sept 12, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 31, 2003
posts:2280
votes: 0


If this problem's been around so long it's a shame someone didn't press Google on it pre-IPO ;)
6:51 pm on Sept 12, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
posts:2141
votes: 0


IMHO:

In the future, Google is unlikely to comment on this issue/problem(?)/bug(?) (or any other), since any "official" comment could be used in support of legal action, whether frivolous or justified.

In my experience, Kerrin's canned response saying "we'll pass this on to our engineers" (in message 41) is an indication that Google is taking the problem seriously and exploring resolutions.

Google needs to first decide if this is their problem or a shortcoming in the HTTP protocol specs that everyone is expected to conform to. Then they must decide if this is a spider problem or an indexing problem. Then they need to fix the appropriate code and test the fix thoroughly before deploying it.

If it's a spidering problem, it likely means the improperly redirected links will need to be respidered, then reindexed. An indexing problem COULD be fixed more quickly if indexing is done independently from spidering I suppose. Google may have a fix in place now and we may be simply waiting for respidering/reindexing corrections to percolate into the publicly available SERPs.

Each of you needs to report your experience with this problem to Google. This page [google.com] suggests reporting it to webmaster@google.com. They need solid examples of the problem to analyze so they can implement a fix. Reporting it or complaining about it at WebmasterWorld does little to improve the situation, although it is an opportunity to vent and commiserate with others experiencing the problem... ;)

9:42 pm on Sept 12, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 1, 2004
posts:65
votes: 0


Google will not be motivated to correct this problem until (a) they receive bad press about it in a mainstream publication, or (b) it directly affects one of their top clients.

wink wink nudge nudge. . .

10:04 pm on Sept 12, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 17, 2002
posts:1187
votes: 6


I think this is happening to mysite. Here's the scenario:

-----

www.widgets.com - has text on homepage describing widgets and stuff. In particular there is a line: "Widgets are the new gudgeon clips."

Searching on google for that line should only find my widgets site but also listed is:

www.foo.com/all-about-widgets/gudgeon-clips.html

The google entry for foo shows my text description but clicking the link goes to the foo site which shows nothing about widgets or gudgeon clips at all.

--------

Is my site being hijacked like marcello's? How can I probe foo.com to see if there is a meta redirect?

From what I gather, when a site is hijacked then google will drop the site and PR0 it.

If that is the case then how quickly does that happen?

Is my site in the early stage of being dropped by google?

10:43 pm on Sept 12, 2004 (gmt 0)

New User

10+ Year Member

joined:Dec 9, 2003
posts:22
votes: 0


Why aren't you all using this link?

[google.com ]

Digital Millennium Copyright Act

It is our policy to respond to notices of alleged infringement that comply with the Digital Millennium Copyright Act (the text of which can be found at the U.S. Copyright Office Web Site, [lcWeb.loc.gov ]) and other applicable intellectual property laws, which may include removing or disabling access to material claimed to be the subject of infringing activity. If we remove or disable access to comply with the Digital Millennium Copyright Act, we will make a good-faith attempt to contact the owner or administrator of each affected site so that they may make a counter notification pursuant to sections 512(g)(2) and (3) of that Act. It is our policy to document all notices of alleged infringement on which we act. A copy of the notice will be sent to a third party who will make it available to the public.

Etc...

11:20 pm on Sept 12, 2004 (gmt 0)

New User

10+ Year Member

joined:Sept 12, 2004
posts:4
votes: 0


Last I checked, Google is a private entity. They can make their search engine operate in any manner they choose. The fact that their algorithm operates in a manner that is not to some of your personal likings is a rather stupid basis for a lawsuit.

Google does not have any kind of duty or obligation to index anybody's website. If they decide they only want to show results for domains that begin with the letter 'A' and were registered on a Tuesday, that's their right. If you don't like how they operate, make your own search engine.

That said, I think this is a pretty major flaw in their algorithm and I, for one, would like to see more extensive analyses of the problem and how it could be resolved. And I'd really be interested in seeing something from GoogleGuy explaining why they do things this way. Enough whining about lawsuits, let's get back to discussion of the actual problem itself.

11:45 pm on Sept 12, 2004 (gmt 0)

New User

10+ Year Member

joined:June 27, 2004
posts:34
votes: 0


as far as i remember that was altavista's words few years ago... they also had rare updates, good amount of advertising and so on... :)

now, vista is sold. thats pity, yeah? :)

12:20 am on Sept 13, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 28, 2004
posts:192
votes: 0


A lot of people are asking about how to look at the redirects or meta tags people are using to possibly hijack your site, so here's the answer for anyone with access to a *nix account -- at the prompt, type:

lynx -mime_header HIJACKURL

where HIJACKURL is the url of the potential hijacker, including "http..." Lynx will show you the HTTP headers, so you can tell if it's a 301, 302, or meta refresh.

12:56 am on Sept 13, 2004 (gmt 0)

Full Member

10+ Year Member

joined:Aug 22, 2003
posts:333
votes: 0


The DMCA doesn't apply in this case because the websites in question aren't copying content, just linking in a way which causes trouble with the way Google indexes pages.
1:24 am on Sept 13, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 13, 2003
posts:1233
votes: 1


The DMCA may not apply but google are obliged to answer one anyway.These are deliberate attempts to pass of another persons work as ones own.
A legal suit may have no effect on google but it should have on the offending culprits and the hosts of such.I would threaten both.
2:53 am on Sept 13, 2004 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1483
votes: 0


>>keep this thread open until GG comes here and tell us what Google will do about it.

Ahh, the good old days.

3:10 am on Sept 13, 2004 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Feb 25, 2004
posts:984
votes: 45


GoogleGuy would show a lot of credibility if he would make a comment.
4:18 am on Sept 13, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:42
votes: 0


A New Day ... But the saga continues!

TO Frank_Rizzo:
Search Google for "wannabrowser".
under "Agent Selection" use "NetSpider"
UN-Check "Follow Redirects"
Leave "Show HTTP Response Headers" Checked

Yesterday I have send a DMCA-complaint to Google and to Altavista by Fax concerning this matter ... hopefully I will get an answer to this fax.

Today all my more than 8,000 backlinks to my site "www.widget.com" have disappeared.
entering link:www.widget.com now gives as answer:
"Your search - link:www.widget.com - did not match any documents."

So now the hijacking-page must have been completely accepted as "THE PAGE" and as Frank_Rizzo says in message 81, the next step will be that my PR6 will become PR0, resulting in a complete loss of:
- a 4-year old site
- over 80,000 pages
- PR6 ranking
- over 30,000 uniques/day
- 200,000 pageviews/day

All of the above the result of someone adding the following line of code to a not so high-ranking page:
"<meta http-equiv="refresh" content="0; url=http://www.widget.com/">"

Also Today (its morning here) My pages in the Google-Index have now dropped from over 80,000 pages to less than 40,000 pages. (using site:www.widget.com)

Also Google traffic is 50% less than the normal average from the last 6-months

I am watching Yahoo like a hawk as I am still getting a lot of traffic from them, but the hijacking-page is STILL NOT not in the Yahoo-Index and my www.widget.com page is still ranking No.1 for its main topic (keywords) on over 3-million results returned.

I still believe in Google and agree with "DaveAtIFG message 79" .... I just hope Google knows about the problem so that other webmasters never have this scenario happen to them.

This 389 message thread spans 26 pages: 389
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members