homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 42 message thread spans 2 pages: 42 ( [1] 2 > >     
301 permanent redirect
using the .htaccess
spud01

10+ Year Member



 
Msg#: 14412 posted 5:45 pm on Jun 17, 2003 (gmt 0)
I have read post like ==> www.webmasterworld.com/forum3/6486.htm and gleaned info on how to go about changing the old url showing i google to the new url.

My .htaccess has
# Send a redirect from our old file to our new file
Redirect index.htm http://www.not-a-real-url.com/index.htm

and it's placed in the old sites root directory.
The redirect works fine, and I also tried what annej
said add <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> , but I still get
--> Status: HTTP/1.1 302 Found
- from the tool found here http://www.searchengineworld.com/cgi-bin/servercheck.cgi

When wouldlike to get '301 Moved Permanently'

Any have any ideas?

 

Birdman

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 14412 posted 6:44 pm on Jun 17, 2003 (gmt 0)

You can add the optional [status] tag.

Redirect index.htm h*tp://www.not-a-real-url.com/index.htm [R=301]

Or you can use RedirectPermanent, which is automatically a 301.

RedirectPermanent index.htm h*tp://www.not-a-real-url.com/index.htm

If you moved a whole site to a new domain and kept the directory and file structure the same you can redirect to each old page's counterpart on the new server.

RedirectMatch (.*)$ h*tp://www.anotherserver.com$1 [L]

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 14412 posted 7:49 pm on Jun 17, 2003 (gmt 0)

The [R=301] flag (and others) applies only to mod_rewrite [httpd.apache.org]. For the Redirect and RedirectMatch directives of mod_alias [httpd.apache.org], the status must be placed immediately after the directive, and consists of either a numerical server response code, or one of: permanent, temp, seeother, or gone.

To redirect "index.html" to another domain using a 301-Moved Permanently redirect, use one of:

Redirect 301 /index.html http://www.example.com/index.html
Redirect permanent /index.html http://www.example.com/index.html
RedirectPermanent /index.html http://www.example.com/index.html

To redirect all files:

RedirectMatch 301 ^(.*)$ http://www.example.com$1
RedirectMatch permanent ^(.*)$ http://www.example.com$1

HTH,
Jim

tschild

10+ Year Member



 
Msg#: 14412 posted 8:49 pm on Jun 17, 2003 (gmt 0)

Actually since the Redirect directive matches a substring from the beginning, the last-named solution (using RedirectMatch) can also be implemented as

Redirect permanent / [example.com...]

This way e.g. /index.html gets redirected to [example.com...] /foo/bar.html gets redirected to [example.com...] etc.

spud01

10+ Year Member



 
Msg#: 14412 posted 1:06 pm on Jun 18, 2003 (gmt 0)

Thx guys,

TSchild's seems to be the best one as the others give me server errors.

Redirect permanent / [example.com...]

I now get:
Status: HTTP/1.1 301 Moved Permanently

and the old site is hosted on:
Server: Apache/1.3.27 (Unix) PHP/4.3.1 .

That sounds about right?

From what I've gleaned in the forums is that, google will now take '301 Moved Permanently' as a firm change of url and follow to the new url updating the index and removing the old url with the new one.

Correct me if am wrong.

Thx.

DavidT

10+ Year Member



 
Msg#: 14412 posted 1:23 pm on Jun 18, 2003 (gmt 0)

I've found that the 64.68 crawlers will not follow 301 redirects after no matter how many attempts.

shaadi

10+ Year Member



 
Msg#: 14412 posted 1:31 pm on Jun 18, 2003 (gmt 0)

I've found that the 64.68 crawlers will not follow 301 redirects after no matter how many attempts.

Is this really True?

:(

elklabone

10+ Year Member



 
Msg#: 14412 posted 1:59 pm on Jun 18, 2003 (gmt 0)

I've found that the 64.68 crawlers will not follow 301 redirects after no matter how many attempts.

Does this mean that the new-improved Google won't follow 301 redirects?

Here's my problem:

I have a dynamic index page with some random content - featured client, random testimonial...

Google has hit www.myverylongdomainname.com and myverylongdomainname.com and thinks they're two different pages. (because of the random content)

www.myverylongdomainname.com has 18 backlinks... one of which is my Yahoo category listing, another is one from the BBB on a PR7 page.

myverylongdomainname.com has 86 backlinks - this is the domain name we normally promote.

A few weeks ago, I read on here that I could do a 301 from www.myverylongdomainname.com to myverylongdomainname.com and that google would combine the backlinks.

However, after Esmerelda, Google has dropped www.myverylongdomainname.com, but myverylongdomainname.com is indexed, but not showing up in the SERPs for our main keyword. At all. We were in the top 10. I'm assuming that Google dropped www.myverylongdomainname.com but hasn't yet switched everything over to myverylongdomainname.com.

What should I do? Wait? Or try to come up with another plan?

I thought about changing my PHP code so googlebot didn't get the randomly generated content (which has caused confusion for her). Would this be considered cloaking? Googleguy?

Any help would be greatly appreciated.

SebastianX

10+ Year Member



 
Msg#: 14412 posted 3:53 pm on Jun 18, 2003 (gmt 0)

> I've found that the 64.68 crawlers will not follow 301 redirects after no matter how many attempts.

That seems to be true:
[webmasterworld.com...]

hcstudios

10+ Year Member



 
Msg#: 14412 posted 4:15 pm on Jun 18, 2003 (gmt 0)

Is there any alternative to a 301 that the 64bot does like? The temporarily moved one (I forget the number) for example? I have an old site (oldcompanyname.com) that I'm redirecting to the new site (newcompanyname.com) and was counting on 301s to get Google to stop treating newcompanyname.com as duplicate content. I don't really want to take down the pages at oldcompanyname.com until I know that all the other search engines (yes, there are other ones out there!) have finished finding the 301s, but it's starting to sound like I'll have a Google penalty until I do so. Big bummer.

Oh, well -- at least this makes my hosting problems with oldcompanyname.com not feel as important (site goes up, goes down, goes up, goes down -- argh). Maybe I'll be lucky and 64bot will only visit oldcompanyname.com during the down times and any dup content penalty will go away (pleasepleasepleaseplease).

DavidT

10+ Year Member



 
Msg#: 14412 posted 4:25 pm on Jun 18, 2003 (gmt 0)

I 301 redirect www.domain.com to domain.com, virtually all of my off site links use domain.com and I've been very thorough about rooting out any www's in internal links.

64.68 will follow the 301 on robots.txt and sometimes the index page but, except very occasionally, nothing else. I kept a log of Googlebot's visits in early June for one week, it made 500+ requests, about 10 of them resulted in a 200 code. I've dropped from 500 odd pages indexed to 62 in fi.

As of Dominic I came off a whacking great two-month PRO penalty so that may have something to do with the bots reluctance to do what it's told.

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 14412 posted 4:40 pm on Jun 18, 2003 (gmt 0)

Let's not confuse the issue by referring to Googlebots by IP number. It is possible that the IP numbers have recently been swapped around. It is also possible that what we used to call Freshbot is now being used instead of Deepbot. There are several threads here where these apparent changes are discussed.

The crawler function that we used to call Freshbot doesn't usually follow links or redirects, it only looks at URLs it knows about for fresh content. The crawler function we used to call Deepbot follows links and redirects.

If you want to have Google change the URL they use to list your pages, do a 301 redirect, not a 302. A 302 is defined [w3.org] as "Moved temporarily". If you use 302, Google rightly assumes the pages will return to the original URL and won't update their index. 301 is "Moved permanently", and that's what is needed here.

Jim

DavidT

10+ Year Member



 
Msg#: 14412 posted 5:25 pm on Jun 18, 2003 (gmt 0)

Jim, I don't really know how to refer to it other than by IP number. How do I know which function it is assuming, deep or fresh, especially since they come in groups so there are a number of different bots acting at about the same time.

In my case anyway it rarely gets to the point of a 302 error because it requests virtually all pages, I am guessing, with www in the url. 90% or more of its requests end in 301, no further than that.

spud01

10+ Year Member



 
Msg#: 14412 posted 10:46 am on Jun 22, 2003 (gmt 0)

So in all respects, even your best efforts to do things right for search engines to realize your site is now hosted somewhere else under a different URL, we find our selves with a 2 IN 3 chance that our new site is found and correctly announced in the search indexes.

hmmm

spud01

10+ Year Member



 
Msg#: 14412 posted 5:27 pm on Jul 11, 2003 (gmt 0)

How long will google take effect of the new url and abolish the old url?

Also, although I have the 301 perm redirect in place using the .htaccess file, I was wondering if I need to add '301 PermanentRedirect' to the title of each page?

This is something I seen on some sites, which no longer have content.

Neo541

10+ Year Member



 
Msg#: 14412 posted 6:38 pm on Jul 11, 2003 (gmt 0)

although I have the 301 perm redirect in place using the .htaccess file, I was wondering if I need to add '301 PermanentRedirect' to the title of each page?

If you have the 301 set in your .htaccess correctly, you don't even need the old page, it will redirect before you hit it, and you won't even see that page's title.

spud01

10+ Year Member



 
Msg#: 14412 posted 9:20 pm on Jul 11, 2003 (gmt 0)

I was considering this option in aid to help google along in indexing only the new url and drop the old one from its index.

dandaman

10+ Year Member



 
Msg#: 14412 posted 10:42 pm on Jul 12, 2003 (gmt 0)

The URLs I am trying to abandon are dynamic querystring URLs, whereas the non-querystring version of these pages is still valid!

I would be interested what you guys recommend for this. Most solutions seem to have a downside to me:

1. Not processing the querystring anymore means I end up letting the URLs default to their non-querystring version, and that in return means duplicate content all over the place.

2. Leaving the querystring functionality in place and deliberately returning a 301. That serves users who still come to the URLs, but I am afraid this could look highly "unnatural" to google to receive pages like that.

3. Returning a 301 together with some new "Location:". Again, this is somewhat "unnatural", as no one else, who isnt a SEO would bother with querystring-only redirects.

4. Like no 3. but I find it hardly possible in my situation to code the right "rules" to forward people to those URLs with equivalent content on them. I could forward ALL querystrings to the non-querystring version of each script. How would that look to googlebot?

4. Any other way of dealing with the issue, can not think of it?

Help appreciated, thanks.

spud01

10+ Year Member



 
Msg#: 14412 posted 2:58 pm on Jul 18, 2003 (gmt 0)

dandaman,

For dynamic urls ie. shopping carts - I would use a .htaccess that has a 301 permanent redirect. Having the .htaccess in the /cart/ folder would be the case hypothetically speaking.

Where as the static pages, u might want to re-direct them individually in a second .htaccess file to the location where your pages reside.

For example

RedirectPermanent /page1.htm www.url.com/page1.htm
RedirectPermanent /page2.htm www.url.com/page2.htm
RedirectPermanent /page3.htm www.url.com/page3.htm
RedirectPermanent /page4.htm www.url.com/page4.htm
........

I'm a novice at this but I think I got the scenario right in ur situation.

In the second scenario if ur pages resite in the root am not sure if the 1st .htaccess is over-rules by the 2nd.

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 14412 posted 4:42 pm on Jul 18, 2003 (gmt 0)

dandaman,

In order to redirect pages with query strings, you'll need to use mod_rewrite. This is further complicated by the fact that on most servers, query strings are not directly available to RewriteRule directives in the .htaccess context; you have to use RewriteCond. Here's an example:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{QUERY_STRING} ^product=widget&model=123&color=blue$
RewriteRule ^cart.html$ http://www.mydomain.com/cart.html?product=widget&model=123[b]rev2[/b]&color=blue [R=301,L]

This redirects requests for only the specified product, model, and color to the page for the new model.

Now, if you are really worried about how this "looks" to search engines, you could use an internal (non-301) redirect, essentially substituting the new page for the old, but without any indication to the user or robot that this is happening. This would open you up to dup content issues. I don't agree that redirecting a query-string request is something that only an SEO would do - It's something that everyone with a query-based site that's been in service a long time might have to do. Anyway, the code to simply substitute a new page for an old one without telling anyone would look something like this:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{QUERY_STRING} ^product=widget&model=123&color=blue$
RewriteRule ^cart.html$ /cart.html?product=widget&model=123rev2&color=blue [L]

BTW, The first two lines in these examples should occur only once in your .htaccess file, above any RewriteConds or RewriteRules, and you may not need the first one at all.

HTH,
Jim

dandaman

10+ Year Member



 
Msg#: 14412 posted 2:44 am on Jul 20, 2003 (gmt 0)

Thanks, spud01 and jdmorgan, but the URLs I have going are just too many to take them all down individually. Especially, because the querystring var I wanna get rid of appears in combination with other vars.

Cheers, for taking all this apache code down, but unfortunately I run IIS with ASP. I kinda checked and I think that IIS doesnt support forwarding querystring. At that point the querystring is not available yet. Of course, it also is nice how apache uses regExp pattern matching to forward, because what if the new URLs are NOT dynamic (~) anymore!?

I guess what I actually wanted to get at with this, how bad can it be to "condensate" URLs? We believe that this is what we need in our case, since we have been a bit too narrow with our keyword focus on certain pages.

Practically speaking that would result in several (dozens/hundreds of) URLs being "moved permanently" to ONE condensed URL. Any reason why this could be a bad thing for google to choke on I wonder? Probably not ... although it could be mistaken to be doorway usage perhaps, and/or it is not really what the inventors of the 301 wanted that function to be for either, is it? What do you think, guys?

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 14412 posted 3:27 am on Jul 20, 2003 (gmt 0)

dandaman,

> it is not really what the inventors of the 301 wanted that function to be

While your "hundreds of URLs" case is an extreme one, it is indeed what the inventors intended - a way to signal to the client that the requested URL should no longer be used, and that the supplied replacement URL should be used in its stead.

Doorway pages are typically set up with 302 redirects, because the intent is to fool the search engine into keeping the doorway URL and thinking that the page is worthwhile on its own. So yours is not a doorway situation.

I suggest you think of your users while deciding what is best. Ultimately, search engines will "like" what users like, and that is a good thing to consider.

Jim

curryrivel

10+ Year Member



 
Msg#: 14412 posted 3:52 pm on Aug 7, 2003 (gmt 0)

We use this php script for permanently 301 moved files. Of course, your server needs to be php enabled.

The advantage is that you can 301 redirect specific pages. I prefer to use an absolute url (as in the example) each time. We've checked the browser headers, and we get a 301 each time:

<?php
header("HTTP/1.1 301");
header("Location: [my-destination-site.co.uk");...]
?>

As for moving a whole domain, that's for someone else to answer.

kanama

10+ Year Member



 
Msg#: 14412 posted 4:32 pm on Aug 7, 2003 (gmt 0)

google is not following 301 redirects from the htaccess file in my experience

my site was in google for the home page only, i changed to a .co.uk and 301ed the .com, the .com was crawled and dropped instantly and the .co.uk was never picked up and crawled

i submitted manually and it was then crawled much later, but still not listed

I dont think the 301 redirect made google crawl the site, id put it down to the links i have recently created instead, if it was going to crawl the site it would have at least followed the redirect and come back later, which it didnt

why they are not following 301 redirects is beyond me completely, id love to hear a good explanation as to why.

claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 14412 posted 9:55 am on Aug 8, 2003 (gmt 0)

>> (301 ...) a way to signal to the client that the requested URL should no longer be used and that the supplied replacement URL should be used in its stead

That's exactly it. I find it very disturbing to learn that Google does not treat so basic information properly - doing the first part without doing the last part makes no sense. This is certainly not good. Restated for US-citizens (Mountain View, CA comes to mind): This sucks.

OTOH, G as well as the other SE's have always had a hard time with "www." and "non-www." versions, which is somewhat understandable, as it is perfectly okay to use "www.example.com" for one purpose and "example.com" for another. (as well as "xxx.example.com", "zzz.example.com", "xyz.example.com" etc.)

Regardless of this, however, a 301 has a distinct meaning that can and should not be interpreted in any other way than "the exact file you requested is now on this (other) location, and will remain there, so please use the new location whenever you want this (same) document."

/claus

[edited by: claus at 9:59 am (utc) on Aug. 8, 2003]

xlcus

10+ Year Member



 
Msg#: 14412 posted 9:56 am on Aug 8, 2003 (gmt 0)

301 redirects have always worked for me. I've even found them useful for getting Google to quickly crawl new pages...

If Googlebot is heavily crawling one of my larger more popular sites and I want it to crawl a new page on a different site, I quickly turn on a 301 redirect to the new page, Googlebot follows it, indexes the page, then I turn the redirect off.

claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 14412 posted 4:14 pm on Aug 8, 2003 (gmt 0)

Just a little comment on the "www / non-www" issue:

I found this thread where jdMorgan and andreasfriedrich both advises Shadow on using "mod_rewrite" in stead of using "Redirect", "RedirectPermanent" and the like.

The former method (rewrite) makes it possible to do a www/non-www redirect without putting the server into an endless loop which is nice. Jims post #4 (version 2) is the way to do it (just remove "www." from lines 2+3 if that is what you want)

LINK: [webmasterworld.com...]

/claus

dandaman

10+ Year Member



 
Msg#: 14412 posted 7:14 pm on Aug 8, 2003 (gmt 0)

Hey, xlcus.

Interesting what you are doing there. Are you saying you (conveniently) turn on 301 by adding that http header plus a new location header, but at the same time leave the page intact.

How important is it to change the actual html content on the page, e.g. a 301 error page? It would be nicer playing around with the 301 without having to change pages, as the browser is supposed to do the redirect anyway. Correct, or would the google "browser" differentiate?

claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 14412 posted 8:26 am on Aug 9, 2003 (gmt 0)

>> would the google "browser" differentiate?

It will not. Gbot will see exactly the same as any other browser, as the 301 redirect happens at the server level - that is before the page is returned to the browser.

You can have, say, "thispage.html" and 301-redirect it to "that-page.html" without moving or deleting "thispage" from the server - "that-page" will just be shown in stead.

Of course, if you do this, the proper way would be to make a 302 redirect (temporary) in stead of a 301 (permanent). Ideally Gbot would follow a 302 as well, but it seems from the above that this is not certain.

/claus

hetzeld

10+ Year Member



 
Msg#: 14412 posted 9:38 am on Aug 9, 2003 (gmt 0)

Dandaman,

This one has always worked for me:

In file /.htaccess of domain olddomain.com,
RewriteEngine on
RewriteRule (.*) [newdomain.tld...] [R=301,QSA,L]

This will redirect all to the equivalent on the new domain, providing you keep the same directory structure.
The R=301 flag will ensure that the moved permanently header is returned to the caller, and the QSA (Query String Append) flag will rewrite your URI and append the original QUERY_STRING once rewritten.

Dan

PS: I have a few articles about URL rewriting on my site. They could help if you can read french. ;)

This 42 message thread spans 2 pages: 42 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved