Welcome to WebmasterWorld Guest from 54.162.139.105

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Lost in Google

     
6:39 pm on Feb 27, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:June 17, 2003
posts:43
votes: 0


My site has been in 1st page serps for many years. Just a few weeks ago, my listing went from a title display and decription to just this:

www.mydomain.com/
Similar pages

In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.

What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?

This is a very "white hat" site - no tricks at all, just good content...

I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.

Is there anything else I can do? Any ideas of why this happened?

9:33 am on Mar 9, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 5, 2002
posts:413
votes: 0


Some sites may be doing this inadvertently, or google may have a reason to keep them in the index.
The sites I reported had
no adsense
no content
(actually there was some cloaked gibberish content)
a driveby spyware/virus install
11:12 am on Mar 9, 2005 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2001
posts:1162
votes: 3


Great Thread!

We have been hit on mature ( > 4 years ) sites, with traffic down to 2% on some sites.

(all sites white hat educational, tons of unique content, IBLs from radio stations, newspapers, universities, libraries, etc)

In our situation it made more sense to ban Googlebot completely, all Gbot is doing for us at the moment is taking bandwidth and giving nothing in return.

The silence from Google on this subject is what prompted us to take (what some may call over zealous) drastic action.

we have left some sites indexable by Gbot so we can see when/if the problem is resolved.

There are some very interesting posts in this thread (should be on the WebmasterWorld frontpage in my view).

Dazz

12:48 pm on Mar 9, 2005 (gmt 0)

Senior Member

joined:Dec 29, 2003
posts:5428
votes: 0


"The silence from Google on this subject..."

maybe it's working just as they planned it.

1:34 pm on Mar 9, 2005 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 27, 2001
posts:1162
votes: 3


maybe it's working just as they planned it.

maybe it is

maybe they love having their SERPS crammed with content stealing 302 redirects, you never know!

maybe they really believe that sites that link to a site should be more relevant than the site itself!

perhaps we should forget about writing content and concentrate on using dodgy tactics just to help Google SEEM more relevant ;o)

we will see soon enough

5:22 pm on Mar 9, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Sept 24, 2003
posts:318
votes: 0


Just a reminder that 'scraper sites' and '302 hijacking' are not the same subject.
7:14 pm on Mar 9, 2005 (gmt 0)

Junior Member

joined:Feb 1, 2005
posts:61
votes: 0


"maybe they really believe that sites that link to a site should be more relevant than the site itself!"

Interesting comment - Many have said the same thing about MSN Search in various threads.

7:25 pm on Mar 9, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2003
posts:197
votes: 0


maybe it's working just as they planned it.

then lets make it work like they didn't plan it.

copywrite lawsuits and protests, both generating tons of bad press.

7:48 pm on Mar 9, 2005 (gmt 0)

Junior Member

joined:Feb 1, 2005
posts:61
votes: 0


Just tested some links from a major directory (ezilon) - with a header viewer: 302 redirects
9:03 pm on Mar 9, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member zeus is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 28, 2002
posts:3444
votes: 1


OK Im thinking load now - In apache it is possible to deactivat a domain/site, then we will see a 404 for the site, if I then go to google remove tool and type in the redircting page for remove, would that be possible.
1:07 am on Mar 10, 2005 (gmt 0)

Junior Member

joined:Feb 1, 2005
posts:61
votes: 0


<Question_+_Speculation>

I often read of people using the Google Toolbar to 'entice' the googlebot to visit a site/page.

IF this 302 redirect problem is real, and IF Google blindly makes a list of URLs for googlebot to visit based on the Toolbar, and IF someone with a Toolbar visits a directory or other site where the page is using 302 redirects (perhaps to see if said page is using 302s to link to their site), is that person contributing to their own demise - that is, giving the googlebot a list of redirect URLs to visit and index?

</Question_+_Speculation>

1:16 am on Mar 10, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


I just run some HTML header displays on links from Yahoo. Exite, Lycos, etc. and they are all using 302 redirects as well. That leaves mostly Google and MSN not doing redirects.

You cannot simply block all 302s, you will lose all your traffic except Google and MSN unless that's OK.

Sheesh.

2:18 am on Mar 10, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 27, 2005
posts:93
votes: 0


IncrediBill,

Good point. I was hoping nobody brought Yahoo into this, This does seem at first glance to contradict all of what this thread is about. and it was inevitable that a keen eye spotted it. But wait, there is an explanation.

All the more reason that google should rectify its problem with the 302. And certainly remove any undue penalties for duplicate content regarding innocent websites.

I have not seen any damaging yahoo redirects. In fact, Yahoo acknowledged and took steps so I would imagine that their scripting is efficient and done in a proper manner. Not only that, they would use industry standard servers or better with programmers who are aware of serverside issues. Most of the damaging redirects emanate from independent servers where anybody can haphazardly manage them with websites residing in them in the hundreds. Each within the server can push out damaging directives. I don’t think Yahoo is out to cause any damage. In fact, at the moment Yahoo is the only real big noise that has addressed the issue of the 302 redirects.

Back street hosting companies with fancy looking websites that give the impression that an army of server engineers work there providing a 24 hour service are the places to look for badly configured serverside issues. When you try to contact these busy looking hosting companies many are manned by a sleepy single individual who may not even know that googlebot has a problem with 302. Talk to them about this issue and the reply maybe “you what mate”. You would most certainly have knowledgeable people at Yahoo who would reply that they are well aware of the issue and their system is checked out by the best engineers in the business.

I created my server myself and it is old and totally customized to conform to no standard. God only knows how it keeps going, day and night.

I would not expect to see yahoo create a meta refresh and a deliberate attempt to manipulate googlebot. I would however expect them, being a big and trustworthy company, to generate all sorts of things during a click process. This is not an issue and they have every right to run their business they way they want to. This forum also uses a redirect process where needed, I have never come across a problem associated with it, though it may not have been exploited. And who is to say that Yahoo’s method of redirect is not exploitable. So long as website owners are not damaged by it nobody will raise any dust. We may find that there exists a loophole in the Yahoo system, time will tell and maybe some blackhat gurus are already on the job.

For instance, draw results in yahoo if indeed a 302 benefits the target site with a multiple keyword that draws the most results for a given website, submit the results address to googlebot if it benefits your site. We all knew that one years ago so don’t try it, penalty may await and an automatic dismissal of your entire site. Don’t try to do it to a competitor it will not work.

Well, let us also see, looks like there may be cracks in Yahoo’s armour and it gives results for surfers with all sorts of weird looking redirects that ultimately end up at the target site. Put a wedge in the crack, hammer it and it will split in two.

The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.

2:27 am on Mar 10, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 27, 2005
posts:93
votes: 0


Addendum to the above.

Let us not drift from the issue at hand. We are talking about deliberate redirects with meta refresh designed to kill sites, and indescriminate and unintentional scripts using go-php, cgi, asp, tracker2, meta refresh, frameset and other destructive elemets and the manipulation of googlebot to create duplicate content pages.

2:27 am on Mar 10, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 3, 2002
posts:2579
votes: 0


Japanese,

You mentioned a couple of tests such as the inurl: search. Another test is the site: search. The site: search should return only pages that are within your site. However, in the case of my hijacking fiasco, the site: command was, and still is, showing numerous unrelated redirect urls. That means Google is truely thinking those urls are part of my site, despite the fact that the parent domains are completely different. What a mess. What a mess.

Incidentally, I started a thread 4 months ago: [webmasterworld.com...] about this topic. Email after email to Google gone unheard. No resolution, and my site is still hijacked with unrelated urls showing in the site: search. My company name isn't even listed in the top 100 when I search for it. Again, what a mess.

2:32 am on Mar 10, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


The internet takes no prisoners, if Yahoo thinks google is having a problem with the 302, well draw your own conclusions.

I'll draw the same conclusion I draw from Yahoo, Excite, Lycos, Alexa and my own site -

I TRACK CLICK THRUS FOR STATS TRACKING USING REDIRECTS, MANY DO IT, NOTHING MALICIOUS INTENDED

If Google has a bug with 302s, then Google needs to fix the bug, simple.

If you want to light a fire under Google, then I suggest the whole 302 Crew take their case to the Google Investor Forums on any stock trading discussion group starting with Yahoo and raise such a holy stink it creates a panic with the investors. Be prepared to write in laymans terms so any 5 year old with 1 share of stock can understand you, and if you make a big enough stink with enough people shouting about and it makes a blip on the stock THEN the media will pick up on it.

Otherwise, you're just wasting time here preaching to the choir.

Guerilla warfare

'Nuff said.

[edited by: incrediBILL at 2:56 am (utc) on Mar. 10, 2005]

This 206 message thread spans 14 pages: 206