| This 206 message thread spans 7 pages: < < 206 ( 1  3 4 5 6 7 ) > > || |
|Lost in Google|
My site has been in 1st page serps for many years. Just a few weeks ago, my listing went from a title display and decription to just this:
In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.
What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?
This is a very "white hat" site - no tricks at all, just good content...
I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.
Is there anything else I can do? Any ideas of why this happened?
Has Yahoo found a solution? And if so, wouldn't they have an interest in sharing it with everybody else? My sites still show up in Yahoo.
Just curious – has anyone ever reported a hijacking problem to Google and got a reply back from them?
"reported a hijacking problem to Google and got a reply back from them"
I think in 1 of the previous redirect-hijack threads, people have reported receiving a standard (your're in our supplemental index) type reply.
Problem is everytime a hijacking site goes into supplemental results, ten new ones will appear, now apparently using cheap info domains. Cloaking different pages for search engines and for visitors.
Adsense should qualify sites. No revenue for them means no motivation to create scrapper sites. But these sites are also Adword customers paying 3c cheapo rates to get your visitors to their scrapper directory. They will entice you with misleading ads just to get you to visit their directory with high paying, completely unrelated page with no way out other than clicking on the links or closing the browser. If you close the browser, you get something like 2 - 3 popups.
For those who think this kind of traffic can convert well, you're kidding yourself.
|I'm saying that Google knows where it landed it was told where to go by the redirect. |
I was thinking the same thing.
What is the reason Google treats 301's differently from 302/meta-refresh? Shouldn't meta-refresh/302's cause the page containing the redirect to be devalued, in the same way they are with 301's?
I appreciate your effort, I will write to GoogleGuy myself and try to explain the problem.
Your input here is fundamental, please lay out any thoughts you have on possible solutions.
There appear to be various types of redirects going on, many of which cause loss of business to web site owners.
We as a group of Webmasters need to lay out the terms and define clearly and concisely the techniques being used by unethical webmasters so that Google and other search engines can focus their efforts on finding a solution to the "bug".
Last night I went to bed early because I had a headache from what I discovered, so I only had a chance to quickly write down 24 web sites that all belong to the same "network", but I'm sure the number is well into the hundreds. I'm not talking about just one site pagejacking another, I'm talking about a network of sites hijacking entire industries.
In this particular case there is a deliberate effort to cash in on quick money by stealing the hard work of others like ourselves who have spent countless hours optimizing sites for our clients.
This individual (or individuals) has a template which acts as if it were a search engine - providing listings for nearly everything imaginable.
Besides the redirect there is also a deliberate effort to "appear" to be legitimate by pretending to link directly to each site listed (when you pass over the listing with your mouse you see the statis bar link).
So to sum things up here, this has become a big business and the 24 domains that I've jotted down are just the tip of the iceberg for this one particular "network", I can't help but think of how many other "networks" there are or will be unless the issue is addressed by the search engines.
Feel free to sticky me for info on this or if anyone has similar cases in study.
Thanks Bobby and Japanese for your kind words.
I will try to list up what I have done in the fight of this hijacker/redirecting problem, but as you know with no success.
I have send emails to googleguy, and a few others, also send emails to adsense and asked them if they are realy interested in sponsoring scraper sites that hijacks other sites.
Used toolbar to get googlebot back
Updated the site
Loaded the whole site to server again to get a fresh date.
Wrote to hijackers and hosting, mostly no reply, hosting dont know what Im talking about.
Added the last part of the hijacker URLs in a do not crawl robots txt and used the removal tool.
I guess now most of you are going out to buy a good rope, paper and pen :), be cause there are no more sane solutions then wait.
One thing I have notised, that with all those hijacker/redirecting the omitted results has also gone nuts, it is as if google have no real content left in there searches and different content is also listed as omitted, but then the hijacker comes in, maybe so many sites are hurt now, that automatic we get omitted results so early even if it is different contents, if thats the case google are in HUGE trouble, finiancial and future.
about the 302 meta that googlebot knows where its going, my theory is that googbot in a way ignores the meta but still goes to the other site, but does not include that it is going to another site.
Do you think that one can get out of this situations if he would buy for exampe a PR7 link to my homepage, in this way his home would have more relevance and it could be considered the good one..
Google's total dominance and total control of the internet with its inability to overcome directive 302 scripts and its capacity problems spells the end of the internet as being a place to host websites.
I traced down today a website that caused a 302 directive, supporting the redirect is a particularly nasty META REFRESH of "0" seconds to the hijacked website.
This is concrete evidence that Zeus’s and my theory is bang on target.... googlebot is being manipulated and I know exactly how it is done.
The only way forward is to attract google’s attention to these posts. Maybe someone higher up than their suave representative here, maybe a guy that understands what we are talking about here.
A CGI PEARL SERVERSIDE INCLUDE DIRECTIVE SCRIPT that is activated by a HTML link on a webpage is not a page that exists. This type of link is only a syntax on the page and it is a “time-bomb” waiting to explode in the face of the unsuspecting website that the link is pointing to.
THE DEADLY WEAPON
This is where googlebot becomes a deadly weapon against unsuspecting sites. A blackhat webmaster places a link to his competitor on a page he knows is visited by as many robots as possible, he makes a few variants to expedite the acknowledgement of the bot regarding the bait link, googlebot follows the link to the page he planted the CGI PEARL path. Googlebot follows the serverside directive of the baited link, it is told the unsuspecting website is at a temporary residence and to go and GET it. Googlebot interrogates the apache server at the “temporary address”, it is presented with a 200 GET, this class of status code indicates that the client's request was successfully received, understood, and accepted the page is new, BRAND NEW according to googlebot regarding the link it followed. But the directive has no content like title tag and description. Googlebot now has created a virtual page to index using the content of the unsuspecting legitimate site, but under the URL of the blackhat webmaster.
It is then that the monster is created, googlebot is acting as a page generator....and google’s so called patented duplicate page detector goes into action when the legitimate site is found containing the same content. Though googlebot does not write html pages, it is using existing pages to attach to hijacking URL scripts. google cannot demote virtual pages, they do not actually exist so nothing to demote, so the page that contains the actual substance is demoted in rankings.
A script such as; [badguy.com...] is not a page or existing website. This is where the big misunderstanding is occuring. A robot like googlebot creates it as a page with the contents of the legitemate site, then attaches the culprit URL that is now a fully functional index result in google. But googlebot can only create the monster only after being instructed by the scripts serverside information.
If this explanation is not clear enough for google and msn, then I think everybody should lose hope and adopt my method of retaliation against host providers that harbor these kind of scripts. I have seen yesterday, 4 CACHED IDENTICAL PAGES 3 of them were indexed as belonging to serverside generated URL's that never existed until googlebot followed and generated the pages listed in google. The other 1 was the legitimate site penalized for duplicate content and in total oblivion in the results of google.
I challenge google and msn or anybody to refute or deny the existence of the above. I have detailed header results for all 4 from 6 different header interrogation tools that show exactly the same results.
Don't purchase any links. I believe one thing and one thing only.
FAIR PLAY TO ALL
Unnatural links will do you damage. A link within a paragraph from another site in anchor keywords is a mega link. Especially if an acronym tag is deloyed with the keyword title.
Google is the embodiment of the American Dream. A company that emerged out of a smoke filled garden garage inhabited by two sleepless looking t-shirt and jeans wearing computer freaks to produce the biggest database the world has ever known for all of us to enjoy.
Let us work together to help them overcome this gargantuan task with an elagant, more refined solution than yahoo did. Many websites using these scripts are not aware they are doing damage.
Keep msn at bay and yahoo in the distant horizon.
|Just curious – has anyone ever reported a hijacking problem to Google and got a reply back from them? |
Yes, I have several clients who had various forms of redirects affecting their sites. I wrote Google about this in mid Nov 04, sending them the links involved in the redirects, and got the following reply:
Thank you for your note. Google aggregates and organizes information
published on the web; we don't control the content of these pages. In this
instance, we suggest that you directly address the webmaster of the page
in question. For more information about our Terms of Service, please visit
If you encounter sites that are trying to deceive our web crawlers, please
submit a report at [google.com...] We use
these reports to collect data that our engineers use to devise scalable
solutions to fight spam in our search results. While we do not always take
action on individual sites as a result of these reports, please be assured
that we are using the information to make large-scale improvements to our
We appreciate your assistance in maintaining the quality of our search
The Google Team
I have written the webmasters/ site owners and hosting companies (which is often the owner of the site) often to no avail but sometimes it works. None of my clients are involved in affiliates so that's not a problem but I had mistakenly submitted most of them to Alexa which I will be correcting shortly..
My biggest problem lately is the hosting company doesn't respond either and their host is themselves so I run into a dead end.
DMCA is the next step.
I THINK I AM GOING TO GIVE UP
Here is an example of very, very bad way that wbsites are causing googlebot problems. Simply follow the below instructions and you will be enlightened as to how big this problem is and it is getting worse by the minute, as you are reading this post many sites are being demoted in google.
1, Go to [alexa.com...]
2, search for COTTON BUDS
3, in the organic results right click on the title link,
4, copy the SERVERSIDE DIRECTIVE URL. In other words what an normal user may call a link.
5, Interrogate the header of the link
6, You will see that it is not a normal link and in fact it is a 302 temporary redirect protocol to robots.
7, The script is harmless when clicked by a human user, ie; browser.
8, The directive script becomes a NIGHTMARE to the website owner if the link is presented to a robot such as googlebot.
9, If you want your competitor to go down in history, now you know how easy it is done.
10, If a link exists anywhere on the net that is accessible to googlebot of the result page produced in alexa, googlebot can follow all of the redirect scripts within it and """"""""BANG""""""" goes your website into total oblivion in google searches.
YOUR WEBSITE WILL NEVER RECOVER without devine intervention.
Correction to the title of my above post.
I THINK I AM GOING TO "SPEW UP"
And don't forget
""""""""" BANG """"""""" goes your website if a competitor lays a linking path for googlebot to find your site in the results page of alexa. Suposedly an ethical website providing a community based website. In fact it is a killing machine that is going out of control with the help of google.
I believe google provides the results clean and alexa plants the deadly timebomb directives into your title links.
""""""""" BANG """""""" that is all it takes.
since many of these problems are causing websites to disappear for thier company/brand name, why not follow the louis vuitton example, and get an offshore friend to buy a few adwords for your brand then file a copywrite lawsuit against google, one has already been won, and more are in the pipeline. Maybe when these issues are harmful to users, webmasters *and* stockholder's bottom line something will be done.
One thing is sure we have to do something and it does not seem we can do anything webmaster related with out breaking rules, which Im not willing to do.
What can we do? we need a responce from google, that would let us know that in the next update everything is clean again, not we are working on it.
I have just contact alexa, told them to remove my links or how to.
I also got a reply from adsense, that they where glad for my email about a site using 302 meta and adsense, they will forward, but I dont think we see any solution.
I remeber the good times where Google said "YOU CAN NOT BE HURT FROM OUTSIDE SIDES"
Did you see the links alexa are using?
I just cannot get them to reply to my e-mails detailing my concerns.
Alexa does not have the right to include your website with SERVERSIDE BASED DIRECTIVES ALTERING THE DYNAMICS OF THE LINK FROM A PURE HTML TO A DEADLY 302 DIRECTIVE PROTOCOL that tells robots your site belongs to alexa.
No reply from them, not even a sausage.
Google must remove its supply of results from them otherwise it will have catstrophic effects. It is happening now.
(I did read your posts in the other threads too.)
If I do a google search allinurl:alexa redirect I get a total of 17 results, mostly url only. Same answer if I query 3 other DCs. Where are all of these alexa redirect urls in the google index?
Back in December, 2004 Yahoo presented an overview of how 301/302 redirects would be handled - here's the pdf document
Note the difference between intra-domain and inter-domain 302 handling. I suspect google is following the same general model.
If you have evidence that the google index is filled with these alexa redirects, please indicate how to find them via a search query. Thank you :)
I just wanted to add I am yet another casuality of this hijacking.
I've been following these posts for a while and have been scratching my head how I could have suddenly dropped out of the search engines.
I just don't know where to go.
Sadly there is not much information out there on the Internet on what to do when this happens to your site. I know my site personally is a redirect 302 hijacking.
However, I also know my site is not getting spidered correctly anymore. I do post and update my site daily also. Luckily, I don't just have one site....
Im in contact with adsense, I have just send a email and asked them if they where interested in a list of scraper sites that are sponsored by adsense and use hijacking or redirecting scripts.
If they say yes I will post it here and you can sticky me all the links, that does not follow adsense rules and hurt your ranking (redirecting), but wait until I have got a reply and I will also first ask brett if its ok.
I also ended some talks with a hosting company that simply refuses to contact a site of them to stop the redircting, it said it has to be that way.
Was just on my way to submit my competitors' 302 redirects from Alexa (er, I mean... just kidding...) and I noticed Google's submit page is not working. Maybe they ARE reading these forums after all and decided to keep any more pages from being submitted. Any time I try to re-submit my pages which have disappeared, to Google through their submit URL pages, I get a quick page redirect, followed by page not found error. Or has someone tried to steal Google's site now? <lol>
Hey Japanese, why not drive the point home once and for all by using your above described technique to hijack the G site. Then they would finally understand what everyone has been trying to tell them and have no other choice than to fix the problem!
Many thanks for your input.
Yahoo’s much heralded and eagerly expected presentation of its 301 and 302 overview failed abysmally to create a sensation that was expected by passionate webmaster who were agog with anticipation of a new directive to govern and standardize such redirects. In fact I have just seen it for the first time myself, with your compliments, and it gave me an anti climax feeling which I was trying to avoid since it was published.
I am none the wiser for seeing it and in no doubt about its insignificance to the internet community as a whole. It looks more like an alphebet training supplement than a hard core anti blackhat hijacking directive.
It must have given google and msn a similar feeling and as yet no pomp and pageantry is planned for the unveiling of google’s answer to these redirects. This problem with redirects is the biggest thing on the internet and it has been dexterously kept under wraps by 2 of the worlds most dominant internet companies. This kind of thing normally brings a company to its knees, but google has so many fans that they just want it to find a solution. When it does, we can then equate the sensation compared to Yahoo’s. Who knows, google’s answer regarding the redirects may be heralded as the new dawning and the definitive revelation of scriptures that will govern redirects and that all who use the internet will one way or another abide by it.
A typical alexa link
The browser or robot is told there, at the serverside, where to go with a 302 directive. And I could not find anywhere in the header or at the alexa site to suggest that googlebot should KEEP the target page. No reassurance of any sort except a frighteningly efficient redirect system.
Can you please assure us that the above script is totally harmless.
What assurance and where can we find evidence that such 302 redirects are approved by google.
I think my site may have been hit by this trick since I am getting no traffic from google since the weekend.
I am getting traffic from the other engines.
Looking at the results I see various redirects but one guy frames by site and you cant see how hes doing the redirect since he loads another php file to load it up.
Looking at the headers I do see a 302 redirect.. but can someone double check for me?
|If I do a google search allinurl:alexa redirect I get a total of 17 results, mostly url only. Same answer if I query 3 other DCs. Where are all of these alexa redirect urls in the google index? |
I've never seen these in referral urls but if you have your site listed in Alexa just copy their link to your site and then run it through a server header checker and you'll find a 302 redirect. I did this to all 25 of my clients this morning and found a redirect on every one of them and then bombarded Alexa with requests for them to be removed.
Sadly there is not much information out there on the Internet on what to do when this happens to your site. I know my site personally is a redirect 302 hijacking.
I have written an extensive article on this. Just search for "hijacking web page pr" and my article should come up in first restults. I have 25 clients and deal with this issue constantly. I checked one client's allinurl results this morn and found 4, 302 redirects and wrote them letters. Got one response so far and they are removing the link--so they say.
I don't do this very often however -- puts me in a
VERY bad mood!
Many thanks for your contribution.
I have a high regard for your thoughts regarding this matter.
I raised the alexa issue because it has been 10 days since I first contacted them to explain why they have adjusted the pure HTML links provided by google for their results into 302 redirects.
They have yet to reply after 3 more requests demanding an explanation as to the efects of the scripts regarding googlebot and how googlebot will behave when presented wIth the 302 redirects. Will googlebot consider the target page be kept intact with no influence of the redirect?
All I wanted was assurance that these redirects are approved by google.
I wanted to then confirm this with google.
I emailed Google a few days ago about my site disappearing from the index. I explained to them that when I searched for my site by domain name, that only the URL displayed and that my description and title were missing.
I also pointed out to Google that when I searched using allinurl: other sites were displayed using my sites URL, title, description, and keywords, in their URL. I gave Google a list of these offending URL's and said that i suspected my site was being hijacked.
Google’s reply is below (names and URL omitted):
Hi Blah Blah,
Thank you for your note. We apologize for our delay in responding to your email.
We searched for your site and found that it is currently included in our search results. To see the results of our search, please visit the following link:
http:// Blah Blah Blah Blah Blah
The Google index contains two types of pages: fully indexed and partially indexed pages. Your page is currently partially indexed. Because our robots were unable to completely review its content during our last crawl, your site appears without a cached copy or detailed title. Instead, it's listed by its URL.
We understand the frustration this situation may cause you. We're always working to increase the number of fully indexed pages in our index. You may be able to improve this page's visibility in our search results by ensuring that a number of high-quality sites link to it. While we can't guarantee that pages in our search results will always be fully indexed, "crawler-friendly" pages have a greater chance of being fully indexed.
Guidelines for creating a "crawler-friendly" site are available at http:// Blah Blah Blah Blah Blah
The Google Team
Sorry to be so dense. I do not have as much experience as all of you.
Is this what you were calling the Serverside directive? [redirect.alexa.com...]
So, if I go to one of the many directories listing my site, and I right click on title, hit properties and I see redirect, then that means I have been hijacked?
Thanks for all of the excellent information.
P.S. I went to alexa and typed in my website without www or .com, many of my pages came up with this,
is this a "A DEADLY 302 DIRECTIVE PROTOCOL "?
i also did a friends and he has the same thing except he has not lost keywords
so i'm still confused.
Lorel has an extensive review of hijacking. Just do a google search and you will find it 1st at "hijacking web page pr".
It absolutely spells out a hijacked site.
I think all of us have the highest regard for google. We are letting off steam in this contained forum because of our frustration that google has not yet found an answer.
If we make enough noise, then we may see a result. Please contribute your thoughts, and yes, as I stipulated, many sites are being hijacked as we speak.
We know the simplicity of hijacking a site, I have never attempted it nor would I advise anybody to take advantage of googlebots ability to generate a result from a dynamically presented link that is only a link for humans browsers but becomes a deadly link when googlebot follows it.
ANYBODY THAT UNDERSTANDS THE BELOW CONTENTS PLEASE CONTRIBUTE., I may be wrong and hope to be contradicted for the sake of people losing their hard work and ranking in google.
Think of it this way; I will put it in laymans terms as possible.
1, A single page exists on the internet. www.cdaaa.com
2, You place a serverside cript as a link www.cdaaa.com/go-php?=www.cdaaa.com%2f for googlebot to follow on that page. Ever since bots were able to read these scripts they have been exploited. It is the only page that exists so you cannot put it anywhere else. It simply indicates to go to the go-php redirector, the rest is crap info to make it look good or can contain affiliate click counter info. It can be made to be a mile long, you can even put a joke in there or say hello to your victim.
3, googlebot follows the link to get instructions from the serverside php redirector and picks up a 302 telling it that the page has temporarily moved and exists at this location “www.cdaaa.com” (this is the killer and culprit, manipulating googlebot) I hope somebody who understands the serverside workings join us to explain exactly how googlebot behaves during this split second procedure. It is vitally important and I believe here rests a starting point.
4, Googlebot proceeds to confirm the domain exists and goes to GET, after a global tour and ends up at the same server is presented with a GET 200 page exists status code. Is presented with the page. In this case the page that contained the kick off link, googlebot takes a snapshot of the page it finds to index in google results. (Hang on a minute, it took a snapshot? Of what?)
5, Now there are 2 pages on the internet.
Clicking on the second link goes first to the website that that contains the serverside 302 directive then you are taken to where the page location exists (remember! The killer). I am confused, will somebody please help me figure this one out. Clicking on the first one goes directly to the page the script link exists.
googlebot penalizes one or the other for duplicate content.
But how can a page that does not exist be penalised?
The only vulnerable page is the real macoy.
6, You e-mail google asking how comes 2 identical pages exists of my index page.
7, google replies;
Our robots crawl the internet on a regular basis. We find new links that point to websites and update on a monthly basis. New websites will be indexed in our next update. Rest assured that no outside influence can jeopardize your ranking in google.
8, Hang on a minute, somebody did just that.
9, You go to use googles URL-CONSOLE to remove the offending link because it ranks above the original. You are told the page exists and cannot be deleted.
10, You make a mistake at the console. Your site is deleted from the index for 90 days.
11, The offending link now remains as the only page on the index. It is alive and kicking because its lifline only resides at the PAGE LOCATION in the go-php redirector (the killer instruction).
Japanese, that's a darn good explanation of what's happening, now we need some senior members like Brett or GoogleGuy to jump in here and contribute their thoughts to the discussion.
I am not a programmer so my next question may seem silly to some, but then great inventions have often come from silly mistakes...
Can a site owner ban an IP address from accessing his/her web site?
For example, I know that Joe HiJacker uses IP address ab.cd.ef.gh, can't I simply tell my server "don't return html" for that IP address?
Once the hijacker cannot display my results as if they were his then I would assume that my site could safely avoid the "duplicate" filter phenomenon.
If enough people start "protecting" their web sites then it might begin to get costly for the hijackers to change IP addresses.
just a thought...
Come on senior members and knowledgable programmers, what is practical here? It may seem Utopian to expect Google or other search engines to find a solution but if we arm the masses we may be able to protect ourselves.
I keep seeing all these woeful stories of people pages gone in the toilet, but I STILL don't understand one thing:
Why does Google credit the originator of the referral?
Based on content alone, it only makes sense that Website #1 is authoritative about blue widgets. Website #3 would get some credit for mentioning blue widgets and the authority. What in Google's logic gives Website #2 any credit at all?
Website #1 (original webmaster): www.bluewidgets.com
|Welcome to Bluewidgets.com! |
Here's our page about blue widgets to read about. _____________________________________________________________
Blue widgets are blue.
Website #2 (hijacker): www.badguy.com?url=www.bluewidgets.com
(or substitute 302, frame-jacking, etc)
|blank page |
Website #3 (other innocent webmaster): www.redsprokets.com
in response to IP based protection. the ip of the website is not the ip you'd be worried about, it would be google's spider IP and delivering different content to them is a no-no.
you could check for referrer and if it matches a hijacking page you could display a different page or nothing at all, but this could be interpreted as cloaking.
| This 206 message thread spans 7 pages: < < 206 ( 1  3 4 5 6 7 ) > > |