Some sites may be doing this inadvertently, or google may have a reason to keep them in the index.
The sites I reported had
(actually there was some cloaked gibberish content)
a driveby spyware/virus install
We have been hit on mature ( > 4 years ) sites, with traffic down to 2% on some sites.
(all sites white hat educational, tons of unique content, IBLs from radio stations, newspapers, universities, libraries, etc)
In our situation it made more sense to ban Googlebot completely, all Gbot is doing for us at the moment is taking bandwidth and giving nothing in return.
The silence from Google on this subject is what prompted us to take (what some may call over zealous) drastic action.
we have left some sites indexable by Gbot so we can see when/if the problem is resolved.
There are some very interesting posts in this thread (should be on the WebmasterWorld frontpage in my view).
"The silence from Google on this subject..."
maybe it's working just as they planned it.
|maybe it's working just as they planned it. |
maybe it is
maybe they love having their SERPS crammed with content stealing 302 redirects, you never know!
maybe they really believe that sites that link to a site should be more relevant than the site itself!
perhaps we should forget about writing content and concentrate on using dodgy tactics just to help Google SEEM more relevant ;o)
we will see soon enough
Just a reminder that 'scraper sites' and '302 hijacking' are not the same subject.
"maybe they really believe that sites that link to a site should be more relevant than the site itself!"
Interesting comment - Many have said the same thing about MSN Search in various threads.
|maybe it's working just as they planned it. |
then lets make it work like they didn't plan it.
copywrite lawsuits and protests, both generating tons of bad press.
Just tested some links from a major directory (ezilon) - with a header viewer: 302 redirects
OK Im thinking load now - In apache it is possible to deactivat a domain/site, then we will see a 404 for the site, if I then go to google remove tool and type in the redircting page for remove, would that be possible.
I often read of people using the Google Toolbar to 'entice' the googlebot to visit a site/page.
IF this 302 redirect problem is real, and IF Google blindly makes a list of URLs for googlebot to visit based on the Toolbar, and IF someone with a Toolbar visits a directory or other site where the page is using 302 redirects (perhaps to see if said page is using 302s to link to their site), is that person contributing to their own demise - that is, giving the googlebot a list of redirect URLs to visit and index?
I just run some HTML header displays on links from Yahoo. Exite, Lycos, etc. and they are all using 302 redirects as well. That leaves mostly Google and MSN not doing redirects.
You cannot simply block all 302s, you will lose all your traffic except Google and MSN unless that's OK.
Good point. I was hoping nobody brought Yahoo into this, This does seem at first glance to contradict all of what this thread is about. and it was inevitable that a keen eye spotted it. But wait, there is an explanation.
All the more reason that google should rectify its problem with the 302. And certainly remove any undue penalties for duplicate content regarding innocent websites.
I have not seen any damaging yahoo redirects. In fact, Yahoo acknowledged and took steps so I would imagine that their scripting is efficient and done in a proper manner. Not only that, they would use industry standard servers or better with programmers who are aware of serverside issues. Most of the damaging redirects emanate from independent servers where anybody can haphazardly manage them with websites residing in them in the hundreds. Each within the server can push out damaging directives. I don’t think Yahoo is out to cause any damage. In fact, at the moment Yahoo is the only real big noise that has addressed the issue of the 302 redirects.
Back street hosting companies with fancy looking websites that give the impression that an army of server engineers work there providing a 24 hour service are the places to look for badly configured serverside issues. When you try to contact these busy looking hosting companies many are manned by a sleepy single individual who may not even know that googlebot has a problem with 302. Talk to them about this issue and the reply maybe “you what mate”. You would most certainly have knowledgeable people at Yahoo who would reply that they are well aware of the issue and their system is checked out by the best engineers in the business.
I created my server myself and it is old and totally customized to conform to no standard. God only knows how it keeps going, day and night.
I would not expect to see yahoo create a meta refresh and a deliberate attempt to manipulate googlebot. I would however expect them, being a big and trustworthy company, to generate all sorts of things during a click process. This is not an issue and they have every right to run their business they way they want to. This forum also uses a redirect process where needed, I have never come across a problem associated with it, though it may not have been exploited. And who is to say that Yahoo’s method of redirect is not exploitable. So long as website owners are not damaged by it nobody will raise any dust. We may find that there exists a loophole in the Yahoo system, time will tell and maybe some blackhat gurus are already on the job.
For instance, draw results in yahoo if indeed a 302 benefits the target site with a multiple keyword that draws the most results for a given website, submit the results address to googlebot if it benefits your site. We all knew that one years ago so don’t try it, penalty may await and an automatic dismissal of your entire site. Don’t try to do it to a competitor it will not work.
Well, let us also see, looks like there may be cracks in Yahoo’s armour and it gives results for surfers with all sorts of weird looking redirects that ultimately end up at the target site. Put a wedge in the crack, hammer it and it will split in two.
The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.
Addendum to the above.
Let us not drift from the issue at hand. We are talking about deliberate redirects with meta refresh designed to kill sites, and indescriminate and unintentional scripts using go-php, cgi, asp, tracker2, meta refresh, frameset and other destructive elemets and the manipulation of googlebot to create duplicate content pages.
You mentioned a couple of tests such as the inurl: search. Another test is the site: search. The site: search should return only pages that are within your site. However, in the case of my hijacking fiasco, the site: command was, and still is, showing numerous unrelated redirect urls. That means Google is truely thinking those urls are part of my site, despite the fact that the parent domains are completely different. What a mess. What a mess.
Incidentally, I started a thread 4 months ago: [webmasterworld.com...] about this topic. Email after email to Google gone unheard. No resolution, and my site is still hijacked with unrelated urls showing in the site: search. My company name isn't even listed in the top 100 when I search for it. Again, what a mess.
|The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions. |
I'll draw the same conclusion I draw from Yahoo, Excite, Lycos, Alexa and my own site -
I TRACK CLICK THRUS FOR STATS TRACKING USING REDIRECTS, MANY DO IT, NOTHING MALICIOUS INTENDED
If Google has a bug with 302s, then Google needs to fix the bug, simple.
If you want to light a fire under Google, then I suggest the whole 302 Crew take their case to the Google Investor Forums on any stock trading discussion group starting with Yahoo and raise such a holy stink it creates a panic with the investors. Be prepared to write in laymans terms so any 5 year old with 1 share of stock can understand you, and if you make a big enough stink with enough people shouting about and it makes a blip on the stock THEN the media will pick up on it.
Otherwise, you're just wasting time here preaching to the choir.
[edited by: incrediBILL at 2:56 am (utc) on Mar. 10, 2005]
My dream of being a orchestral conductor has at last come to fruition. I enact my catharsis at the webmasterworld forum. Everybody is singing off the same hymn sheet exept you.
What are we going to say to the chap who is in a hopeless situation one post above. people like crobb305 needs help and answers, you and I at least know that google should react positively.
I sympathise with you. I too have what were once MEGA COMMERCIAL SITES. Able to sustain 6 wage earners. They were hit by hijackers. I cleaned them up but they have not recovered. And no, they are not sandboxed. They were penalised for duplicate content.
However, I closed one hijacker down, crippled another and am now enjoying much at the expense of a few.
Please continue with your efforts to google. They are in 75% control of their robot the other 25% is in the hands of blackhat webmasters. My share is insignificant.
I want to add...that I have been working on getting the 302s removed. Over the past 6 months, I have been able to remove 30 to 40 redirects using the Google url removal tool and by setting the robots meta tag on the destination page to noindex. Now, all of the redirects are gone. The urls still showing in the site: search that I mention above no longer redirect to my site, so I can't have them removed using the removal tool. Unfortuantely, gbot hasn't touched those urls since Nov 2, thus it does not know they no longer redirect to me.
So, even though all redirects are gone, the penalty remains. How long will it stick? My hijacking problems started last May. The last redirect was removed 2 months ago. Among the redirects I removed were about 10 tracker2 urls, all of which were showing in the site: search as recently as December. Regardless of what you do to clean up the problem, the penalty sticks, indefinitely.
|What are we going to say to the chap who is in a hopeless situation one post above |
"Meet me in the GOOG stock chat room @ 8am EST fri morning when the 302 Mob invades"
Starting a revolution only requires 1 shot.
Good idea. Although the goal here is not to bring down Google. They simply need to be more respectful of the hardwork diligent webmasters do to bring content. By ignoring our concerns and brushing this problem under the rug they are saying "we don't care about you and we don't care about the quality of our serps". But hey, business is business. And, if they are going to ignore us, then maybe it's time to take the issue mainstream.
Google management could simply take notice of these concerns and deligate a group to resolve the problem asap but they chose not to.
[edited by: crobb305 at 3:12 am (utc) on Mar. 10, 2005]
You are correct and sound adroit at cleaning up those redirects.
However, google is not going to address this problem or give an explanation if we give up. We must fight together for an acceptable answer.
My theory is that google got themselves in a big mess with their adwords campaign. They were too loose with applications to host google adds.
Rectification of the 302 issue for google so soon after going public at the stock exchange will bring the company to its knees. I guarantee this because an enormous change in results will occur and it will be unpredictable.
Stocks will exchange hands fast and devaluation of google must be avoided at all costs at the moment and foreseeable future.
Yahoo will soon compete with its own, more refined version, this will also begin to deplete google’s market share because many angry webmasters will change over.
I personally hope google finds a solution and keeps the others at bay. So we are trying to help google find an answer, they are just being secretive.
Perhaps we need to organize a 302mob, I'd suggest a mailinglist but I don't want to be seen as promoting for another site here. If anyone is interested in discussing organizing something please sticky me.
|Good idea. Although the goal here is not to bring down Google. |
Who said anything about bringing down Google?
This has obviously been a problem in the minds of webmasters that has gone unresolved for over a year. When so many webmasters have supposedly contacted Google over this issue and they either ignore or give lip service to individual emails, the only last result is to make it public. One could say posting about it in WebmasterWorld is making it public, but the only public here is your likewise helpless web geek peers.
When I say make it public, they are now a publicly owned company, and their continued silence on this issue seems to force the issue to the only forum to get a public answer which is in front of shareholders and traders. Just beware how you do this as retributions could be nasty, I'd hate to see sites vanish or AdSense accounts cancelled in retaliation, or worse.
However, before making a HUGE stink, it would be nice to hear someone like GoogleGuy weigh in on whether this is in fact a real problem or just a perceived problem. It would give everyone a black eye if it was just misguided paranoia that doesn't actually reflect the true reason the redirecting sites that are ahead of your site, they may be ahead of you for other reasons.
If everyone suffering from this perceived problem wrote them at once, hundreds or more, they might take notice. Give them a week to respond and if they ignore you all, then hit them publicly.
[edited by: incrediBILL at 3:56 am (utc) on Mar. 10, 2005]
|Just a reminder that 'scraper sites' and '302 hijacking' are not the same subject. |
Yeah, I am confusing the two together.
Hmm... what happen if there are alot of 302 redirects on a single page? Which of the redirected pages will it takes over in google.
No, it doesn't. They've done tremendous damage to who knows how many web-businesses because of their near-monopoly, courtesy problems they've know about for many, many months. They didn't give a sh*t because they were all sitting around in piles of money from the IPO. If you think G rocks, you must also be a big fan of M$.
I can't help but think it's in Google's best interest to at least recognize the problem here with us webmasters and work together to find a solution.
I don't understand the silence.
The "silence" simply means they are brushing the bug under the rug, so-to-speak.
They have known about the 302 problems for well over a year. They brag about the number of "pages" indexed, yet many many of these are useless redirect urls.
"In google search box INURL:YOURSITE.COM
If this result produces strange looking url's like [hijacker.com...] "
i did that and guess one page i found in 1 out of 12 results!
well how about that is also Google hijacking or there new technology(algorithm) has a serious problem.
My conjecture is that G knows (and cares) about this issue, but is trying to find an algo type solution to it. Expect great fanfare and praise when the problem is finally solved. (a problem they don't even admit- presently!)
Wow, when I posted a thread in November about this, I was unsure of the real extent of this problem. But now there are two simulatenous threads with 100+ posts. Plus, numerous threads stemming back to January 2004 and even earlier. Among those threads:
[webmasterworld.com...] (Jan 2004)
Judging from the number of threads/posts and the number of times this topic has made front page news at Webmasterworld, it is clear the problem is enormous. Google certainly knows of the problem, and it is growing to affect thousands of great content pages.
[edited by: crobb305 at 8:13 am (utc) on Mar. 10, 2005]
The best way I've found to find a 302 to your url is to search for some unique text from your page in quotes. The offending page will show up and you can view the google cache of the page to see if it matches yours.
| This 206 message thread spans 7 pages: < < 206 ( 1 2 3 4  6 7 ) > > |