Forum Moderators: Robert Charlton & goodroi
www.mydomain.com/
Similar pages
In addition, my Yahoo listing disappeared as well. I then did a Yahoo search for pages with my domain included and found most of my interior pages indexed but not my home page.
What happened? Is it possible my site was not ready to be crawled when Googlebot and Slurp robots visied my site - simultaneously?
This is a very "white hat" site - no tricks at all, just good content...
I went and manually requested my site be spidered on both Google and Yahoo, and sent an email to Yahoo requesting any explanation as well.
Is there anything else I can do? Any ideas of why this happened?
I’m amazed this isn’t the topic of every Web newsletter on the net this week, but it isn’t. We’ve all heard the old saying “that will never happen to me”. After seeing a friend go through this a few months ago I foolishly said “that will never happen to me”. Well it has and I can assure you if your website or sites are good which I suspect they are it will happen to you in the very near future if it hasn’t already.
The ball is in Goggle’s court to do something. We can suggest ideas all day long however
It’s up to them to act. There’s power in numbers so please take five minutes of your evening tonight to drop Google a quick e-mail telling them how your site has been ruined by hijackers or if you’re one of the lucky ones who hasn’t been hijacked yet your concerns of being hijacked. Like I said earlier I never thought it would happen to me, but it did and I can’t describe to you the sick feeling I’ve had since I discovered my site was hijacked. Act now or act later it’s up to you.
Now it's past midnigt here, so i don't really have time to write a lot, other than that japanese is right in most of the stuff about how this works. There are some minor tech details and some wording issues, but by and large he's got it 100% right.
(please don't use so many capital letters, though, it hurt my eyes)
Also, IncrediBILL is right above - there are legitimate referrers and it would not be good banning those, as that way nobody could link to you. It's a very tricky situation and there's no apparent solution that the individual webmaster can use. I do know my .htaccess stuff ;)
It must be fixed by Google - they are the ones that have created this problem by indexing "pages" that don't exist [1]. So, they should fix it.
Before i'm off to bed, i will just point to this thread i started in May 2004: What about those redirects, copies and mirrors? [webmasterworld.com]
I have been following this and related issues since sometime in 2003, and they have not been fixed yet.
---
[1] Note: You're right here as well, japanese. I have used that term myself before as well. I'll post the specifics later.
Does anyone know if G has a problem indexing Doublely 301 redirected pages?
MikeNoLastName: You may be confusing one problem for another. I don't think the redirect from domain.com to www.domain.com is going to cause any problems. However, the permanent redirect to newdomain.com (assuming newdomain.com is really a brand new, just registered domain) is probably going to cause problems with the so-called 'sandbox' problem.
The sandbox problem prevents new sites from ranking for their terms, even though the site may be in the index. Your permanent redirect is essentially saying "dont go do olddomain.com/page.htm anymore, since it's moved permanently to newdomain.com/page.htm". Google removes oldomain.com/page.htm from the index, spiders newdomain.com/page.htm, and then puts it in the sandbox.
Matt Cutts had said at one of the conferences that Googlebot can follow multiple redirects fairly well, as long as they aren't four or five levels deep. I think you've got the sandbox working against you.
The only good script is a script that bears the hallmark of google’s approval.
I never asked google to approve my redirects, didn't know they needed to be consulted on this.
Alexa must disclose whether their redirecting method is indeed been approved by google.
I'm at a loss for words, hard to think while cachinnating
Thanks for the heads-up on About.com, just dropped in a frame buster script on my site to stop that nonsense.
Many thanks for joining this thread. Your input will be much valued here.
I tried to join webmasterworld with the nickname of altavista but a .htaccess file called Brett presented me with a 403 access denied. So I changed my nickname to Japanese and was presented with a 200. I hope I am not seen here as a Trojan and marked for deletion..
I have read some fabulous posts in many different threads here but this problem with googlebots ability to generate pages when it encounters particular 302 redirects is probably going to dwarf the sandbox issue.
I think that it already has and more and more people are seeing their sites collapse into total oblivion at great expense to them. Very little reference is ever made to any other search engine. Google is the internet and many loyal fans are not happy.
I suggest we think in terms of how to arm ourselves as webmasters to prevent this happening on a local level to each and everyone of us personally.
Last night I had a friend of mine who is a programmer come over and we talked about a whole range of solutions which we will begin testing tonight, all of which can be implemented by a webmaster either on your site or server.
I think it is fundamental that we share ideas regardless of how silly they may seem.
In my particular case it is not just 1 web site that has a redirect pointed at 1 of my web sites, it is a network of web sites (at last count 50) that are deliberately using hidden redirects - even to the point of changing the statis bar to appear to point at the sites whose content they are stealing - in order to take advantage of the many hours of optimization so they can rocket to the top for every conceivable search possible.
In most of the cases they have their throwaway domains all on 1 server with similar IP addresses, I know their names and the name of who has the server. Can this information be used to "report" them or "denounce" them so they can no longer do so?
For example, could there be a "black list" (either by IP or or registrant) which can be verified by Google?
As far as a localized solution is concerned, how about implementing a script on our web sites that goes something like this:
If referral comes from X, or if referral string is longer than Y return 404. The reason why length of referral string is important is because the hijackers are using a long string to hide the redirect from the spider.
Actually what we are considering is returning the exact copy of the referrer string so that it creates a loop, but I'm not sure what effect it will have on the server I use.
googlebot doesn't carry referrers
Stargeek are you suggesting the script wouldn't work for Google? I think you are right. Google will continue to index my content as if it were the hijacker's content. The idea of the script however is to make it so the users have problems with the hijacker's sites, making it a problem for them to have a redirect to my site.
Finally Google is banning sites again. If they're gaming the system, then they'll have to pay the price.
During the weekend, I sent an e-mail to a spammer:
"Spammer:"
"Please eliminate ASAP the following link:
spammer.com/cgi-bin/se/go.cgi?id=MY_SITE"
"Otherwise, I'll contact Google to report the address
spammer.com"
"You have 48 hours to eliminate the link."
Well, all links to MY_SITE were eliminated within the 48 hours.
Nevertheless, it's good to know Google does take action against spammers via reports.
Google rocks!
