Forum Moderators: Robert Charlton & goodroi
www.mydomain.com/
Similar pages
In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.
What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?
This is a very "white hat" site - no tricks at all, just good content...
I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.
Is there anything else I can do? Any ideas of why this happened?
We have been hit on mature ( > 4 years ) sites, with traffic down to 2% on some sites.
(all sites white hat educational, tons of unique content, IBLs from radio stations, newspapers, universities, libraries, etc)
In our situation it made more sense to ban Googlebot completely, all Gbot is doing for us at the moment is taking bandwidth and giving nothing in return.
The silence from Google on this subject is what prompted us to take (what some may call over zealous) drastic action.
we have left some sites indexable by Gbot so we can see when/if the problem is resolved.
There are some very interesting posts in this thread (should be on the WebmasterWorld frontpage in my view).
Dazz
maybe it's working just as they planned it.
maybe it's working just as they planned it.
maybe it is
maybe they love having their SERPS crammed with content stealing 302 redirects, you never know!
maybe they really believe that sites that link to a site should be more relevant than the site itself!
perhaps we should forget about writing content and concentrate on using dodgy tactics just to help Google SEEM more relevant ;o)
we will see soon enough
Interesting comment - Many have said the same thing about MSN Search in various threads.
maybe it's working just as they planned it.
then lets make it work like they didn't plan it.
copywrite lawsuits and protests, both generating tons of bad press.
I often read of people using the Google Toolbar to 'entice' the googlebot to visit a site/page.
IF this 302 redirect problem is real, and IF Google blindly makes a list of URLs for googlebot to visit based on the Toolbar, and IF someone with a Toolbar visits a directory or other site where the page is using 302 redirects (perhaps to see if said page is using 302s to link to their site), is that person contributing to their own demise - that is, giving the googlebot a list of redirect URLs to visit and index?
</Question_+_Speculation>
You cannot simply block all 302s, you will lose all your traffic except Google and MSN unless that's OK.
Sheesh.
Good point. I was hoping nobody brought Yahoo into this, This does seem at first glance to contradict all of what this thread is about. and it was inevitable that a keen eye spotted it. But wait, there is an explanation.
All the more reason that google should rectify its problem with the 302. And certainly remove any undue penalties for duplicate content regarding innocent websites.
I have not seen any damaging yahoo redirects. In fact, Yahoo acknowledged and took steps so I would imagine that their scripting is efficient and done in a proper manner. Not only that, they would use industry standard servers or better with programmers who are aware of serverside issues. Most of the damaging redirects emanate from independent servers where anybody can haphazardly manage them with websites residing in them in the hundreds. Each within the server can push out damaging directives. I don’t think Yahoo is out to cause any damage. In fact, at the moment Yahoo is the only real big noise that has addressed the issue of the 302 redirects.
Back street hosting companies with fancy looking websites that give the impression that an army of server engineers work there providing a 24 hour service are the places to look for badly configured serverside issues. When you try to contact these busy looking hosting companies many are manned by a sleepy single individual who may not even know that googlebot has a problem with 302. Talk to them about this issue and the reply maybe “you what mate”. You would most certainly have knowledgeable people at Yahoo who would reply that they are well aware of the issue and their system is checked out by the best engineers in the business.
I created my server myself and it is old and totally customized to conform to no standard. God only knows how it keeps going, day and night.
I would not expect to see yahoo create a meta refresh and a deliberate attempt to manipulate googlebot. I would however expect them, being a big and trustworthy company, to generate all sorts of things during a click process. This is not an issue and they have every right to run their business they way they want to. This forum also uses a redirect process where needed, I have never come across a problem associated with it, though it may not have been exploited. And who is to say that Yahoo’s method of redirect is not exploitable. So long as website owners are not damaged by it nobody will raise any dust. We may find that there exists a loophole in the Yahoo system, time will tell and maybe some blackhat gurus are already on the job.
For instance, draw results in yahoo if indeed a 302 benefits the target site with a multiple keyword that draws the most results for a given website, submit the results address to googlebot if it benefits your site. We all knew that one years ago so don’t try it, penalty may await and an automatic dismissal of your entire site. Don’t try to do it to a competitor it will not work.
Well, let us also see, looks like there may be cracks in Yahoo’s armour and it gives results for surfers with all sorts of weird looking redirects that ultimately end up at the target site. Put a wedge in the crack, hammer it and it will split in two.
The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.
Let us not drift from the issue at hand. We are talking about deliberate redirects with meta refresh designed to kill sites, and indescriminate and unintentional scripts using go-php, cgi, asp, tracker2, meta refresh, frameset and other destructive elemets and the manipulation of googlebot to create duplicate content pages.
You mentioned a couple of tests such as the inurl: search. Another test is the site: search. The site: search should return only pages that are within your site. However, in the case of my hijacking fiasco, the site: command was, and still is, showing numerous unrelated redirect urls. That means Google is truely thinking those urls are part of my site, despite the fact that the parent domains are completely different. What a mess. What a mess.
Incidentally, I started a thread 4 months ago: [webmasterworld.com...] about this topic. Email after email to Google gone unheard. No resolution, and my site is still hijacked with unrelated urls showing in the site: search. My company name isn't even listed in the top 100 when I search for it. Again, what a mess.
The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.
I'll draw the same conclusion I draw from Yahoo, Excite, Lycos, Alexa and my own site -
I TRACK CLICK THRUS FOR STATS TRACKING USING REDIRECTS, MANY DO IT, NOTHING MALICIOUS INTENDED
If Google has a bug with 302s, then Google needs to fix the bug, simple.
If you want to light a fire under Google, then I suggest the whole 302 Crew take their case to the Google Investor Forums on any stock trading discussion group starting with Yahoo and raise such a holy stink it creates a panic with the investors. Be prepared to write in laymans terms so any 5 year old with 1 share of stock can understand you, and if you make a big enough stink with enough people shouting about and it makes a blip on the stock THEN the media will pick up on it.
Otherwise, you're just wasting time here preaching to the choir.
Guerilla warfare
'Nuff said.
[edited by: incrediBILL at 2:56 am (utc) on Mar. 10, 2005]
