Google's response to: Redirected URL

[Maia]

Hello.

I have a site about a town in a country. My site was listed in a directory which only included links that were related to that country. The directory was a subdirectory of a site about that country, containing tons of great content. The directory uses 301 redirect links. The link looked something like this:
www.[fake site].com/dir/links/123 and allowed them to show how many clicks you had from it.

Recently I was checking some of my keywords in Google. I found some high-ranking results which contained the title and snippet from my home page with www.[fake site].com/dir/links/123 instead of my URL. Is that normal for a redirected link?

I didn't know if this was OK, so I sent an email to Google to ask them. It took them a long time to reply. In the meantime, I tried to do some research. From what I understood, since it was redirected it shouldn't be a duplicate content issue. I checked the backlinks on the fake-site URL-and the backlinks were the links that linked to my site.

Today I got an response from Google. They suggested I contact the owner of the other site and ask them to remove the redirect.

The other site has removed my listing (and made sure to mention they blocked my IP from adding any more sites to their directory).

Does it sound like that site was up to something sneaky?

[edited by: ciml at 11:08 am (utc) on Sep. 3, 2004]
[edit reason] De-linked. [/edit]

[Maia]

Maybe my post was too confusing.

What I am trying to figure out is...if a directory redirects a link to my site:

How does it end up in Google with my homepage's title and description and the directory's url?

If it was a 301 redirect, why did Google tell me to have them remove the link?

I can PM you an example of what I am talking about if you don't understand.

Thank you.

[spharalsia]

Had this happen to me a while back. See this thread for more information:

[webmasterworld.com...]

About a month after the offending site removed the link (at my request), I wrote to G to ask them to fix the listing in the SERPs. A few weeks later the listing disappeared.

...Which is really too bad, since the fake listing that was redirecting to me had a higher position in the SERPs than my "real listing" did.

Good luck!

[Maia]

Spharalsia,

Thanks for the link! That was exactly the info I was searching for.

[Boaz]

I know there have been a few reports of redirect 301 not working perfectly, but at least in my (extensive) experience with redirect 301, I've always seen it work perfectly with Google, Google showing the landing page and not the redirecting page. I've seeen this at many different sites of different formats, sizes, and on different servers.
So in my opinion it is quite probable that while that site was doing a redirect, it wasn't a redirect 301.

[vincevincevince]

This effect has been getting a lot of bad press recently. I'd like to add this:

Most searchers for latest widget news know the well-marketed and linked address:
widgetnews.foo/latest

widgetnews.foo, in line with most major media distributers, generates a dynamic URL which points towards a static page for the latest news, allowing better indexing, as well as reducing server load, and could well include a session or other info:
widgetnews.foo/09-06-2004-1035_a454f1.htm

Which would Google be right to index? Will SE users looking for widgetnews latest want to see the long non-descriptive URL which keeps changing, or the nice friendly URL which people always link to?

The model only breaks down when other site use redirects to your site, and they seem a lot more important than your site does to Google (sorry about that), which now considers that your real URL is just where the content is stored, not where it belongs in terms of a user-based hiarachy.

[allanp73]

Maia,

The same thing happened to me. Actually my problem was with Yahoo where as a result my site got banned due to the redirect link. I am now going through the process of getting the site back into their index. The problem is there are many directories using these kinds of links. They mean no harm they do it just to track click through. However, both Google and Yahoo's bots are not sophisticated enough to recognize these links for what they are.
Either these links should be ignored or they should be interrupted as just another link to the site instead the current way the bots understand them which is to see them as duplicate content.

[jdMorgan]

To clear up some points here, the first thing you need to do when considering a listing in a given directory is to use the server headers checker [webmasterworld.com] to see what kind of redirect(s) the directory uses on its links. Examine the output of that utility, and look for the server response code. A 301-Moved Permanently is acceptable. A 200-OK or a 302-Found (Moved Temporarily) is not acceptable. You will get a 200-OK response for pages which contain a meta-refresh (not good). Multiple redirects are not good, but could be acceptable (I don't know - see final pp below), but starting with the listed URL, follow each redirect manually with the headers checker, and note the response code and the URL being redirected to, entering each URL into the headers chacker in succession to "follow" the redirect path.

> there are many directories using these kinds of [incorrectly redirected] links. They mean no harm they do it just to track click through.

Most do not mean any harm, but some do. In Maia's case, it sounds like they did it intentionally to achieve the reported result.

Server response codes have specific meanings to browsers and search engine robots. It is not up to the search companies to redefine these meanings; If they did, things would be a lot worse than they are now. Rather, it is incumbent upon Webmasters to use the correct response codes, and to check that sites that link to their sites also use correct server response codes.

200-OK ------------------------ Index this page content at this URL.  
301-Moved Permanently --------- Go to this new URL and index that page content using that (new) URL.  
302-Found (Moved Temporarily) - Go to this new URL and index that page content using this (old) URL.

Refs:
Negative Effect of Domain Name Forwarding? [webmasterworld.com] (A related problem)
RFC2616 HTTP/1.1 Protocol [w3.org] (See Section 10 - "Status Code Definitions")

If someone who has this problem cares to experiment, I'd like to know what happens if you ask one of these badly-linking directories to link to a page on your site and then use a 301-Moved Permanently redirect to redirect that page to a new page. In other words, if you add a permanent redirect to the end of a redirection chain involving 200-OK and/or 302-Found redirects, does the spider follow all of those redirects and does the final 301 on your own site "fix" the problem?

Jim

[Maia]

Boaz and jdMorgan,

My mistake. They are using 302 redirects to the home pages.

Google has now dropped my homepage from the index.

[Boaz]

Maia, it definitely sounds to me then as though that directory was up to no good. At least they agreed to remove your listing, that's something (assuming they're not cloaking and so telling you they've removed the listing while still presenting this listing to Google).
The bad news though is that it could take quite some time for Google to recognize your home page again (though I've seen "miracles" happen from time to time).

[Maia]

How can I tell if they are cloaking? If you click on the link now, it gives a database error, then goes to a 404 error page.

[Boaz]

What do you mean by "click on the link"? Is the link still present, or do you mean if you type the link in the browser?

Generally you can tell if they cloak by comparing the actual page where the link was to the cache of that page in the Google results (though it may take some time for the cache to be updated to recent changes). If the page and its cache are identical, they probably don't cloak. If they're reasonably identical except for minor changes that could be explained by time differences between the cache and the original, they still probably don't cloak. If there are big differences then they either cloak or they have done major revisions to the page and the cache is not up to date.

[Maia]

I meant when I click on the link in the Google SERPs. The cache hasn't been updated since 8/28 and the directory removed the link on 9/2.

Oh well, lesson learned.

Boaz, thank you for your help.

[McMohan]

Well, I just finished going through one full cycle of this very same problem, just the difference being the directory used a Meta Refresh instead of a 302 to redirect to my site.

About 3 months back, found out a directory that listed my site used a META refresh to redirect to my URL. The redirecting URL had taken on the position of my site. i.e when I search for my URL in Google, it showed the redirecting URL. Ranks dropped on account of that, reported to Google, got a reply too! that they will look into the matter, restored my site URL back in Google, and just yesterday restored the ranks too in their original positions.

Huh! Need I take a break? :-)

Best wishes Maia

Mc

[Maia]

McMohan,

That is awesome Google took care of the problem in your case.

I have exchanged several emails with them, but they don't seem to see any connection between my homepage, and only my homepage, being dropped after the other site appeared in the SERPs with the redirect to my site.

What Google told me is that the changes I cite are consistent with normal fluctuations and not indicative of wrong-doing or penalization of individual sites.

If I didn't get dropped due to duplicate content, I wonder why Google, themselves, told me to ask the other site to remove the redirected link to my site.

I guess they must view meta-refreshes differently than a 302 redirect.

Good luck to anybody else that encounters this.
Maia

[george123]

i suggest we should panish those guys by entirly coping there sites under a new domain or a free website like i did with one of them,i did email them to remove the redirect to my page they didnt ,so i made the next step and copy there entire page put a tone of incoming links,hoping that they will be vanished for good,wait and see.

[paulsilver]

I am wondering if one of you would tell me if links that look like this constitute a redirect of the nature you are describing...

snipped

If you click the link, it first goes to a page in the site, then to the site it is linked to...

These are links that are generated by a link database tool, open source, that a site I know of uses...

I know for certain that the site using this database tool is only using it to count clicks so that it can display the most popular links it shows... they do not charge for the links and dont claim that these are reciprocal links to other sites...

Please advise...

[edited by: DaveAtIFG at 2:47 am (utc) on Sep. 8, 2004]
[edit reason] No specifics please! [/edit]

[helenp]

Does this happens to all redirects 302 or only some?
Actually Iīve been banned by yahoo/Inktomi for long,
recently had my penalty lifted,
and I know one directory that does redirect with 302 to my and other sites.,
It has good pr in google, it is sort of famous directory.
Should I tell them to deleate my listing? just in case?
donīt think they would, though I asked for months ago if I was banned in Yahoo due of this redirecting,
Actually this directory seems to be banned in Yahoo.

[McMohan]

One thing is clear, Google has some room to improve in handling the redirecting URLs. But, this problem doesn't seem to be acroos the board, since the directories that redirect, employ the same method to all the sites they list and only few sites suffer from this problem.

Mc

[webdude]

Okay,

Today I noticed that my site has been hijacked also. The results in the SERPs show my title and the rollover shows my text but the link goes to [widget.com...]

When I check the header at this address, I got HTTP/1.1 200 OK Date: Wed, 08 Sep 2004 15:14:48 GMT Content-Type: text/html . The actual page shows nothing, blank except for a meta refresh to my page. No title, no nothing.

If you do a link:www.widget.com/out.cgi?id=618, it shows EXACTLY all my backlinks as if I did link:mydomain.com

I was able to remove the page through an editing tool (thank the stars).

Now when I checked the header I got HTTP/1.1 302 Moved Date: Wed, 08 Sep 2004 15:18:44 GMT Server: Apache/1.3.29 (Unix) location: [widget.com...] Connection: close Content-Type: text/plain

This clearly now shows a 302 to their main site.

Questions...

How long will it take to straighten this mess out?

Will deleting this link eventually put my page back in the SERPs (it is totally missing now)?

Do I report this to Google?

I have found several directories that are using this method of a meta refresh to my site. Should I report all of them?

Has anybody else gotten any satisfaction from Google on this?

Anyway, thanks for the responses...

[dimka]

I was playing with google redirecting bug... and in my case results were as follows:

>>How long will it take to straighten this mess out?

After G found out that the redirecting page was removed it restored site at previous position at SERPs in 2 days.

>>Do I report this to Google?

Not sure. I just used google remove page tool to speed up the process.

>>I have found several directories that are using this method of a meta refresh to my site. Should I report all of them?

That's the question i ask myself. I think that G actually counts such redirected links. The site becomes hijacked only when the PR of redirecting site is bigger.

[mcavill]

I think I'm being a bit slow here...is the general consensus that 302's are bad, but won't get you booted / your page hijacked, whereas meta redirects serving a 200 do (assuming the redirecting page is considered more important...)?

[Marcello]

Yes, curently your page get booted from Google if it gets hijacked by a 302, 301 or a meta-refresh from another site.

See this latest post about page hijacking [webmasterworld.com]

a lot of this is going on, I lost my main index page to a hijacker, emailed Google days ago, but no word from them yet.