Forum Moderators: Robert Charlton & goodroi
www.mydomain.com/
Similar pages
In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.
What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?
This is a very "white hat" site - no tricks at all, just good content...
I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.
Is there anything else I can do? Any ideas of why this happened?
We have been hit on mature ( > 4 years ) sites, with traffic down to 2% on some sites.
(all sites white hat educational, tons of unique content, IBLs from radio stations, newspapers, universities, libraries, etc)
In our situation it made more sense to ban Googlebot completely, all Gbot is doing for us at the moment is taking bandwidth and giving nothing in return.
The silence from Google on this subject is what prompted us to take (what some may call over zealous) drastic action.
we have left some sites indexable by Gbot so we can see when/if the problem is resolved.
There are some very interesting posts in this thread (should be on the WebmasterWorld frontpage in my view).
Dazz
maybe it's working just as they planned it.
maybe it's working just as they planned it.
maybe it is
maybe they love having their SERPS crammed with content stealing 302 redirects, you never know!
maybe they really believe that sites that link to a site should be more relevant than the site itself!
perhaps we should forget about writing content and concentrate on using dodgy tactics just to help Google SEEM more relevant ;o)
we will see soon enough
Interesting comment - Many have said the same thing about MSN Search in various threads.
maybe it's working just as they planned it.
then lets make it work like they didn't plan it.
copywrite lawsuits and protests, both generating tons of bad press.
I often read of people using the Google Toolbar to 'entice' the googlebot to visit a site/page.
IF this 302 redirect problem is real, and IF Google blindly makes a list of URLs for googlebot to visit based on the Toolbar, and IF someone with a Toolbar visits a directory or other site where the page is using 302 redirects (perhaps to see if said page is using 302s to link to their site), is that person contributing to their own demise - that is, giving the googlebot a list of redirect URLs to visit and index?
</Question_+_Speculation>
You cannot simply block all 302s, you will lose all your traffic except Google and MSN unless that's OK.
Sheesh.
Good point. I was hoping nobody brought Yahoo into this, This does seem at first glance to contradict all of what this thread is about. and it was inevitable that a keen eye spotted it. But wait, there is an explanation.
All the more reason that google should rectify its problem with the 302. And certainly remove any undue penalties for duplicate content regarding innocent websites.
I have not seen any damaging yahoo redirects. In fact, Yahoo acknowledged and took steps so I would imagine that their scripting is efficient and done in a proper manner. Not only that, they would use industry standard servers or better with programmers who are aware of serverside issues. Most of the damaging redirects emanate from independent servers where anybody can haphazardly manage them with websites residing in them in the hundreds. Each within the server can push out damaging directives. I don’t think Yahoo is out to cause any damage. In fact, at the moment Yahoo is the only real big noise that has addressed the issue of the 302 redirects.
Back street hosting companies with fancy looking websites that give the impression that an army of server engineers work there providing a 24 hour service are the places to look for badly configured serverside issues. When you try to contact these busy looking hosting companies many are manned by a sleepy single individual who may not even know that googlebot has a problem with 302. Talk to them about this issue and the reply maybe “you what mate”. You would most certainly have knowledgeable people at Yahoo who would reply that they are well aware of the issue and their system is checked out by the best engineers in the business.
I created my server myself and it is old and totally customized to conform to no standard. God only knows how it keeps going, day and night.
I would not expect to see yahoo create a meta refresh and a deliberate attempt to manipulate googlebot. I would however expect them, being a big and trustworthy company, to generate all sorts of things during a click process. This is not an issue and they have every right to run their business they way they want to. This forum also uses a redirect process where needed, I have never come across a problem associated with it, though it may not have been exploited. And who is to say that Yahoo’s method of redirect is not exploitable. So long as website owners are not damaged by it nobody will raise any dust. We may find that there exists a loophole in the Yahoo system, time will tell and maybe some blackhat gurus are already on the job.
For instance, draw results in yahoo if indeed a 302 benefits the target site with a multiple keyword that draws the most results for a given website, submit the results address to googlebot if it benefits your site. We all knew that one years ago so don’t try it, penalty may await and an automatic dismissal of your entire site. Don’t try to do it to a competitor it will not work.
Well, let us also see, looks like there may be cracks in Yahoo’s armour and it gives results for surfers with all sorts of weird looking redirects that ultimately end up at the target site. Put a wedge in the crack, hammer it and it will split in two.
The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.
Let us not drift from the issue at hand. We are talking about deliberate redirects with meta refresh designed to kill sites, and indescriminate and unintentional scripts using go-php, cgi, asp, tracker2, meta refresh, frameset and other destructive elemets and the manipulation of googlebot to create duplicate content pages.
You mentioned a couple of tests such as the inurl: search. Another test is the site: search. The site: search should return only pages that are within your site. However, in the case of my hijacking fiasco, the site: command was, and still is, showing numerous unrelated redirect urls. That means Google is truely thinking those urls are part of my site, despite the fact that the parent domains are completely different. What a mess. What a mess.
Incidentally, I started a thread 4 months ago: [webmasterworld.com...] about this topic. Email after email to Google gone unheard. No resolution, and my site is still hijacked with unrelated urls showing in the site: search. My company name isn't even listed in the top 100 when I search for it. Again, what a mess.
The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.
I'll draw the same conclusion I draw from Yahoo, Excite, Lycos, Alexa and my own site -
I TRACK CLICK THRUS FOR STATS TRACKING USING REDIRECTS, MANY DO IT, NOTHING MALICIOUS INTENDED
If Google has a bug with 302s, then Google needs to fix the bug, simple.
If you want to light a fire under Google, then I suggest the whole 302 Crew take their case to the Google Investor Forums on any stock trading discussion group starting with Yahoo and raise such a holy stink it creates a panic with the investors. Be prepared to write in laymans terms so any 5 year old with 1 share of stock can understand you, and if you make a big enough stink with enough people shouting about and it makes a blip on the stock THEN the media will pick up on it.
Otherwise, you're just wasting time here preaching to the choir.
Guerilla warfare
'Nuff said.
[edited by: incrediBILL at 2:56 am (utc) on Mar. 10, 2005]
My dream of being a orchestral conductor has at last come to fruition. I enact my catharsis at the webmasterworld forum. Everybody is singing off the same hymn sheet exept you.
What are we going to say to the chap who is in a hopeless situation one post above. people like crobb305 needs help and answers, you and I at least know that google should react positively.
I sympathise with you. I too have what were once MEGA COMMERCIAL SITES. Able to sustain 6 wage earners. They were hit by hijackers. I cleaned them up but they have not recovered. And no, they are not sandboxed. They were penalised for duplicate content.
However, I closed one hijacker down, crippled another and am now enjoying much at the expense of a few.
Please continue with your efforts to google. They are in 75% control of their robot the other 25% is in the hands of blackhat webmasters. My share is insignificant.
So, even though all redirects are gone, the penalty remains. How long will it stick? My hijacking problems started last May. The last redirect was removed 2 months ago. Among the redirects I removed were about 10 tracker2 urls, all of which were showing in the site: search as recently as December. Regardless of what you do to clean up the problem, the penalty sticks, indefinitely.
What are we going to say to the chap who is in a hopeless situation one post above
You say:
"Meet me in the GOOG stock chat room @ 8am EST fri morning when the 302 Mob invades"
Starting a revolution only requires 1 shot.
Google management could simply take notice of these concerns and deligate a group to resolve the problem asap but they chose not to.
C
[edited by: crobb305 at 3:12 am (utc) on Mar. 10, 2005]
You are correct and sound adroit at cleaning up those redirects.
However, google is not going to address this problem or give an explanation if we give up. We must fight together for an acceptable answer.
My theory is that google got themselves in a big mess with their adwords campaign. They were too loose with applications to host google adds.
Rectification of the 302 issue for google so soon after going public at the stock exchange will bring the company to its knees. I guarantee this because an enormous change in results will occur and it will be unpredictable.
Stocks will exchange hands fast and devaluation of google must be avoided at all costs at the moment and foreseeable future.
Yahoo will soon compete with its own, more refined version, this will also begin to deplete google’s market share because many angry webmasters will change over.
I personally hope google finds a solution and keeps the others at bay. So we are trying to help google find an answer, they are just being secretive.
Good idea. Although the goal here is not to bring down Google.
Who said anything about bringing down Google?
This has obviously been a problem in the minds of webmasters that has gone unresolved for over a year. When so many webmasters have supposedly contacted Google over this issue and they either ignore or give lip service to individual emails, the only last result is to make it public. One could say posting about it in WebmasterWorld is making it public, but the only public here is your likewise helpless web geek peers.
When I say make it public, they are now a publicly owned company, and their continued silence on this issue seems to force the issue to the only forum to get a public answer which is in front of shareholders and traders. Just beware how you do this as retributions could be nasty, I'd hate to see sites vanish or AdSense accounts cancelled in retaliation, or worse.
However, before making a HUGE stink, it would be nice to hear someone like GoogleGuy weigh in on whether this is in fact a real problem or just a perceived problem. It would give everyone a black eye if it was just misguided paranoia that doesn't actually reflect the true reason the redirecting sites that are ahead of your site, they may be ahead of you for other reasons.
If everyone suffering from this perceived problem wrote them at once, hundreds or more, they might take notice. Give them a week to respond and if they ignore you all, then hit them publicly.
[edited by: incrediBILL at 3:56 am (utc) on Mar. 10, 2005]
Just a reminder that 'scraper sites' and '302 hijacking' are not the same subject.
Yeah, I am confusing the two together.
Hmm... what happen if there are alot of 302 redirects on a single page? Which of the redirected pages will it takes over in google.
Google rocks!
No, it doesn't. They've done tremendous damage to who knows how many web-businesses because of their near-monopoly, courtesy problems they've know about for many, many months. They didn't give a sh*t because they were all sitting around in piles of money from the IPO. If you think G rocks, you must also be a big fan of M$.
I don't understand the silence.
GoogleGuy? Brett?
They have known about the 302 problems for well over a year. They brag about the number of "pages" indexed, yet many many of these are useless redirect urls.
If this result produces strange looking url's like [hijacker.com...] "
i did that and guess one page i found in 1 out of 12 results!
www.google.com/quality_form?q=related:www. mysite.com/&hl=en&lr=&ie=UTF-8
well how about that is also Google hijacking or there new technology(algorithm) has a serious problem.
any coments?
[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...] (Jan 2004)
Judging from the number of threads/posts and the number of times this topic has made front page news at Webmasterworld, it is clear the problem is enormous. Google certainly knows of the problem, and it is growing to affect thousands of great content pages.
[edited by: crobb305 at 8:13 am (utc) on Mar. 10, 2005]
