http://widgets.com vs http://www.widgets.com

lost my serps

[sanblasena]

HI,

Someone recently linked to my site from 4 other sites (without my permission) and used [widgets.com...] as the links. Now, I have gone from the front page to page 3 or 4 on all my keywords and my site is showing up in google as widgets.com (all the other site have www in front). [widgets.com...] shows a pr 0 with no backlinks.
[widgets.com...] show a pr4 with many backlinks. Any help would be appreciated.

Thanks. Patricia

[Yidaki]

Hey Patricia, put this in your .htaccess file:

RewriteEngine on 
RewriteCond %{HTTP_HOST}!www\.widgets\.com 
RewriteRule (.*) http://www.widgets.com/$1 [R=permanent,L]

This way, pr and link pointers will be transfered to the www version of your site and it will be the only one listed at google.

More info about how and why can be found here:
RewriteRule www non-www [google.com]

[edited by: Marcia at 3:59 pm (utc) on Nov. 16, 2003]
[edit reason] Edit per member request. [/edit]

[sanblasena]

Thanks so much, Patricia

[caveman]

OK, in light of some recent comments from well place sources here, the issue of how to redirect and merge traffic from [site.com...] to [site.com...] has reared its head again, and less technically blessed individuals like me need help.

Forget meta refresh unless there are no alternatives. I'm particularly curious about any expert input related to, for example, setting up a second virtual domain, versus SymLinks approach or other .htaccess solution. I remember reading at least in one place that there were downsides to the sym method.

The goal is to ensure that *one* homepage receives maximum credit for inbound links...and typically that would be the www. version. However since that may not always be so, perhaps any help could cover moving from www. to non-www. too.

Also I'm writing this because I tried the solution noted above and get a 500 error. It could be that I'm just not adding some little obvious character, or deleting one.

TIA/
caveman

p.s. I don't believe for one second that this will solve the missing index problem, but it should be done for any serious website anyway.

[caveman]

Actually I just found another string in WW in which jdMorgan offered up this code, and it worked in my case:

In .htaccess, you may need this first line as well:
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST}!^www\.domain\.com
rewriterule (.*) [domain.com...] [R=permanent,L]

(Note start anchor added to RewriteCond and slash added to rewriterule)

Jim

Jim was right to say that you *may* need that first line because in my case that line was *not* needed for the code to redirct traffic to the www. address...

Still I'd love it if some knowing expert in here would comment on:
Do I lose anything by not having that first line, and also, any thoughts on possible negatives from using this method as far as the SE's go...?

thanks again,
caveman

[ALbino]

I tried Jim's method recently and I end up with [domain.com...] redirecting to [domain.com...] with two /'s. Another example would be [domain.com...] redirecting to [domain.com...] again with the two /'s. Anybody know what causes this/how to fix it?

[closed]

caveman:

Do I lose anything by not having that first line, and also, any thoughts on possible negatives from using this method as far as the SE's go...?

I usually don't add this line whenever a newbie asks for code, because it's almost never needed. FollowSymLinks is usually enabled by default. You only need that line if, for example, you get a 500 error that says that you need FollowSymLinks to be enabled.

So to answer your question(s), no and no.

ALbino:

I tried Jim's method recently and I end up with [domain.com...] redirecting to [domain.com...] with two /'s.

Generally, in .htaccess, the path that's used in patterns does not begin with a slash, but in httpd.conf, it does.

So you should convert the rewrite rule pattern (.*) in .htaccess to /(.*) in httpd.conf.

[Powdork]

If I am redirecting domain.com to www.domain.com do I put this in the .htaccess file for www.domain.com or domain.com.

[jdMorgan]

caveman,

Some servers are set up to enable the FollowSymLinks option in the server config file. Therefore, the Options directive is not needed in .htaccess. In some cases where it is not needed, the server is set up to not allow changes to the options, so having the Options directive can result in a server error if you try to change the Options.

So it all has to do with how the server is set up above the .htaccess level.

ALbino,

closed probably nailed it... If you are installing the rule in httpd.conf, use

 RewriteRule [b]^/(.*)$[/b] http://www.domain.com/$1 [R=permanent,L] or
RewriteRule [b]^(.*)$[/b] http://www.domain.co[b]m$1[/b] [R=permanent,L] 

instead.

Powdork,

The intent of the specific code posted above was to redirect domains in the case where they all were hosted in the same account. So, really in that case, there is only one .htaccess. But if you have separate .htaccess files you would put the code in the account you want to redirect from.

Also, everybody watch out... A space is required between the "}" and the "!" in this line. Posting here on the forums removes that space.

 RewriteCond %{HTTP_HOST} !^www\.domain\.com 

To successfully post the code here, use two spaces.

Jim

[Powdork]

The intent of the specific code posted above was to redirect domains in the case where they all were hosted in the same account.

so if we have a basic paid hosting solution, and when someone types domain.com it gives the same pages as www.domain.com, is it reasonable to assume that there is just the one account?

[jdMorgan]

Yes, I think you'd know if you had two accounts, because you'd have to ftp update two sites, even if they used the same pages.

It sounds like you can use the 'stock' code above. You can use it to go either from www to non-www or from non-www to www. Just make sure the domain names match in both lines for either case, except for the required escaping of special characters (like periods) in the RewriteCond. The reason the lines need to match has to do with the logic of the code:

# If the requested domain was NOT www.domain.com
RewriteCond %{HTTP_HOST} !^www\.domain\.com
# then redirect to www.domain.com using a permanent [b]R[/b]edirect. This is
# the [b]L[/b]ast rule that needs to be processed for the current HTTP request.
RewriteRule ^(.*)$ http://www.domain.com/$1 [R=permanent,L]

Jim

[Powdork]

Yes, I think you'd know if you had two accounts, because you'd have to ftp update two sites, even if they used the same pages.

That makes it crystal clear, Thank You

[Powdork]

will this affect any of the order, denies I have, or better yet, will this still work (given that the other stuff already works)

RewriteEngine on
RewriteCond %{HTTP_HOST}!^www\.domain\.com
rewriterule (.*) [domain.com...] [R=permanent,L]
order deny,allow
deny from ****xx
deny from xxx
deny from xxx
deny from xxx

[pgkooijman]

I use ASP and IIS. I used a 301 redirect set up in IIS to move a site to a new domain. I have not setup 301 redirects for all my other sites. At this moment I have host headers set up for the same site.

Are you implying that I should set up a second site for all my websites with the name widget.com and have that 301 redirect to www.widget.com?

[soapystar]

why cant you just 301 redirect?

[jdMorgan]

Powdork,

Your allow/denies use a different Apache module - mod_access as opposed to mod_rewrite. They really don't affect each other, and can go in any order. I personally put my mod_access stuff first, but it's up to you.

pgkooijman,

No. You don't want to set up another physical server in either case. In IIS, I believe you can accomplish the same thing using Control Panel. Do a search at WebmasterWorld for 301 redirect domain IIS [google.com] or similar, or browse through the previous posts in the Microsoft-Related forum.

The overall idea is that www.domain.com and domain.com are not the same thing. Most clients and many hosts assume that they are or might be, but they are not really the same thing. Therefore, if you have incoming links to www.domain.com and also to domain.com, you are depending on the search engines to 'figure out' that they are really the same sites and to give you full credit for incoming link popularity or for Google PageRank. The 301-Redirect technique is intended to *tell* the search engines, users, and other webmasters that you want to be known by and linked to at one domain and one domain only.

Jim

[killroy]

For simplisity I usually simply set up both, but one with a 301 redirect.
since I never use .htaccess, I leave somebody smarter to the translation.

This is for httpd.conf:

<VirtualHost ip.ip.ip.ip>
ServerName domain.com
Redirect 301 / [domain.com...]
</VirtualHost>

The ip should be the same as for the www subdomain. Of course this depends on your DNS settings since you can set up both for different IPs as well.

SN

[Powdork]

I tried it every way in every conceivable order with the symlinks stuff and without to no avail. Maybe a fresh head in the morn will help.

[ALbino]

Thanks closed and jdMorgan. That fixed my problem.

[pgkooijman]

What do you mean by just 301 redirect? At this moment I know only one method and that is set up a site for the XXX.com and use IIS to set a permanent redirect to www.XXX.com. Are there other ways?

[canuck]

Also, if you use sub-domains make sure you add a RewriteCond for them, so they aren't all redirected to your www depending how things are set up.

Note: This is if you are redirecting www.domain.com ---> domain.com as I do...

[caveman]

jdMorgan,

Thanks. You've helped me before on config questions; always much appreciated.

I still have one remaining question about the symlinks method:

I've read that symlinks opens up the possibility for security breaches.

Does the Virtual Host method (like that noted by Killroy) - defined in a VirtualHost block of your htppd.conf file - offer better security? I can change my .htaccess file easily enough, but think that my hosting company would need to make the virtual host change...

[bunltd]

killroy: Thanks for your blurb about using a Virtual Host to handle 301 redirects - that worked beautifully for me. :)

Here's hoping that Google will be kind now that I took care of that unintentional-bit-of-never-mattered-before-stuff.

LisaB

[Zonka]

I apologize if this is off-topic, my site also had the duplicate www.name.tld as well as name.tld, and since only the name.tld was used thoughout the site, and the www-form was meant as a convenience for users who think that all URLs have to be preceeded by www... I decided to get rid of the www.name.tld version and instructed G to remove it ... As of this writing the removal of the www.name.tld is still marked as pending but the name.tld has vanished out of the SERPs (can't even find it by "http://name.tld") -- is this to be expected or?

[bhartzer]

I used this in the .htaccess file to redirect all of the domain.com to www.domain.com:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST}!^www\.domain\.com
rewriterule (.*) [domain.com...] [R=permanent,L]

The problem is that if you do this it will screw up all your subdomains. It works great, but if you have subdomains like links.domain.com that you don't want redirected then you shouldn't use this because everything will be have a 301 permanent redirect to www.

canuck mentioned writing a rewritecond for the subdomains. What would that look like for links.domain.com (as an example)?

[julinho]

Is there any easy way (by that I mean: a way which requires as little technical knowledge as possible) to check out wether or not one's sites are being affected by this .htaccess, .htconfig, 301, etc?

One of my sites domain.com has inbound links pointing to domain.com, domain.com/, www.domain.com, www.domain.com/, www.domain.com/index.html.

When I type site:domain.com -adfjije (which returns all the pages from the domain), I get exactly the same results as when I type site:www.domain.com -adfjije ;in both cases, all pages listed include the www.
Would that mean that there is already some kind of redirection (as seen by Google) from domain.com to www.domain.com?

Also, when I try site:www.domain.com/ -adfjije or site:domain.com/ -adfjije ,the search doesn't return any matches. Should I worry about that?

More questions: if a .htaccess is missing or misconfigured, shouldn't the consequence be just a case of duplicated content (for, example, domain.com and www.domain.com)? Then, shouldn't google just drop the copies and keep one of the pages (the one with higher PR) listed? How could this misconfiguration result in the complete dropping of the index.html of the Google index?

Thanks in advance.

[jdMorgan]

All,

This needs to be ten threads, not one... :o

pgkooijman,

I really don't know - I'm not an IIS expert. Again, see the link above for MS-IIS-related posts here at WebmasterWorld, or look through the MS-Related subjects forum.

Powdork,

You might want to say again what the problem is.

caveman,

mod_rewrite demands that FollowSymLinks be enabled before it will execute in a per-directory (.htaccess) context. I don't know why, but it does. SymLinks *can* be dangerous if you allow them to point from public areas to restricted files. As in most things, it's all a compromise...

Zonka,

It's not to be expected, but I have heard of that happening before. Your best bet is to not use the "Remove URL" link at Google, but simply 301 redirect from the domain you don't want to use to the one that you want to keep, as we are discussing here.

bhartzer,

Yes, you're right. That code was never intended for use on sites where there are multiple subdomains in use.

You can either exclude the active subdomains from being redirected, or you can specify which subdomains you do want to redirect. The way that code is written, it uses a "catch-all"; If the domain is NOT the right one, then redirect. For sites with active subdomains, that isn't going to work.

julinho,

> Is there any easy way (by that I mean: a way which requires as little technical knowledge as possible) to check out wether or not one's sites are being affected by this .htaccess, .htconfig, 301, etc?

Use the server headers checker [webmasterworld.com].

> One of my sites domain.com has inbound links pointing to domain.com, domain.com/, www.domain.com, www.domain.com/, www.domain.com/index.html.

This is "non-optimal".

> When I type site:domain.com -adfjije (which returns all the pages from the domain), I get exactly the same results as when I type site:www.domain.com -adfjije ;in both cases, all pages listed include the www.
Would that mean that there is already some kind of redirection (as seen by Google) from domain.com to www.domain.com?

> Also, when I try site:www.domain.com/ -adfjije or site:domain.com/ -adfjije ,the search doesn't return any matches. Should I worry about that?

> More questions: if a .htaccess is missing or misconfigured, shouldn't the consequence be just a case of duplicated content (for, example, domain.com and www.domain.com)? Then, shouldn't google just drop the copies and keep one of the pages (the one with higher PR) listed? How could this misconfiguration result in the complete dropping of the index.html of the Google index?

All of the above is a concern because you are allowing/requiring Google to figure out where your site is. You have links to multiple subdomains, and are relying on Google being 'smart' and figuring out that domain.com and www.domain.com are the same. They usually do figure it out. But if you have inconsistent links on you own site, or lots of other inconsistencies, then maybe you confused Googlebot, so your index page got dropped. Who knows! I prefer to 'keep things neat' -- one 'main' domain name consistently used for everything, and legal alternative domain names redirected to the one main domain name. Doing this up front when setting up a new site actually helps prevent people from linking to you at the wrong domain, too.

Jim

[julinho]

Jim,
Thanks for the reply.

About the 'non-optimal' linking: the problem is that many times you can't control the way that people link to you, and also most people (I presume) don't know the difference between these different ways of linking, and just don't care about it.
I will read more about it.

Thanks again.

[caveman]

Thanks Jim, again!

caveman

[caveman]

Oh Cripes, just thought of another question.

OK, I've set up the mod_rewrite to redirect all "non-www" traffic to www.mydomain.com/index.html. Great, done. But not so fast.

Do I also have to do something to rediret www.mydomain.com/ to www.mydomain.com/index.html? It won't work as a simple .htaccess redirect. Do I need another mod-rewrite string?

This is not my favorite part of site operation...

Thanks, anyone still willing to help the dumb :-(