Forum Moderators: phranque
I have a corporate client (for just one of their many sites) with a fair number of domain names for each of several product lines, so we're talking about a lot of domain names. They have domains for everything from misspellings to protecting brand names to territoriality issues ("if we don't have this, one of our competitors will use it")... and they currently have the relevant domains pointed at each product site, in effect creating many mirror sites.
When I suggested that they eliminate the mirrors and use 301 redirects to the main sites, I came up against an argument that I hadn't encountered before. They don't have their own hosting, and they said that it's cheaper to point domains than it is to host them separately and use 301s.
They're incredibly tight with their money, so my standard SEO arguments haven't been enough to persuade them that avoiding mirror sites is worth the extra money... and I know hardly anything about servers, so I couldn't argue with them on those grounds.
Someone from a hosting company I spoke with briefly told me that there is a way to set up the servername aliases, as I think he called them, so they would point to the main site... and then to apply some sort of command forcing the server name to the main domain. I couldn't follow what he was telling me in the time we had. Can anyone explain?
But those would look like identical sites and not redirects to a single site.
Also, if your hosting company set you up with a dedicated ip and did not enable name-based virtual hosts then apache will return the same stuff regardless of the host header in the request. Which also means having identical sites.
Another option is to use .htaccess and mod_alias (that one is simpler than mod_rewrite) and return a 301 response code for all secondary domains with a location on the primary domain.
If you hosting company is telling you that it costs more to have multiple domains pointing to the same document base - dump that company. They are just trying to get more money out of you.
Besides, how many domains do you have? You can get a dedicated server (at least a blade partition) for $90/month and do whatever you want with it.
René.
Make sense?
Yes... extremely clever. Thank you...
It would be nice, though, to further save them the extra $1-2K a year in hosting which that one extra account for each brand would cost. This is a lot better than the $10-20-30K it might have cost them to host each of the aliases separately... but I'll bet they'd think of it still as an unbudgeted expense.
Looking again at my very rough notes from my conversation with the hosting company (which wasn't their hosting company; just a company I was dealing with on another project)... I think the suggestion was to set up each alias in the config file and then put in (and this is what I can't really decipher) a 'hard coded' config line (?) 'forcing the server name to the main domain.'
Further thoughts?
In response to other suggestions... a dedicated server is not in the cards. I don't think the company has personnel to set it up or run it...
I know that Google ultimately just displays only the domain with most links. My feeling, though, is that mirror sites are simply unnaceptable. They lead to incoming link problems down the road... particularly on other engines... and it's something I feel should be cleaned up from the start.
- give server a list of server aliases in the Apache config file
- using the server name directive, tell it what to respond with
I'm told that what should happen is that when aliasdomain.com or whatever is entered, desireddomain.com will be returned.
I'd appreciate elaboration, feedback, and comments.
As I understand the suggestions...
- NFFC is suggesting two IPs per site, one for the main domain and one for the others.
- bcc1234 is suggesting using just one IP per site, to include both main and secondary domains.
- sun818 suggests an IP dedicated to hosting all his redirects (I assume for many sites), and a separate IP for the main domain of each site.
What are the strategic and reliability differences among the various approaches? Why don't NFFC and sun818 do it all on one IP? Is there any advantage to keeping the IP with the main domain free of aliases and redirects?
Also, if your hosting company set you up with a dedicated ip and did not enable name-based virtual hosts then apache will return the same stuff regardless of the host header in the request. Which also means having identical sites.
I assume this means that name-based virtual hosts need to be enabled. While I'm not technical enough to know what this means exactly, I think I understand that what we're discussing (pointing many domain names at the same site) is different from the intent, at least, of "normal" low-rent name-based virtual hosting (where many totally separate sites share one IP address).
The client's web guy, though, has the attitude that a "top-tier" hosting company wouldn't offer virtual hosting, as it's an option I'd suggested for the secondary domains before I posted. Keep in mind that he's not the hosting company. He's the webmaster or whatever for the client.
And I'm not the IT guy in this process. I'm just the SEO who's trying to talk the client out of having mirror sites; and I have such a small vocabulary when it comes to servers that I'm struggling for words to describe the problem.
The discussion that prompted this post came about because the client webmaster felt that he couldn't redirect the domains to the main domain unless they were each hosted separately, and this then became a cost issue.
So... in this case would his attitude about enabling name-based virtual hosts be...
- a legitimate concern?...
- or just a carry over from a bias against low-rent hosting?
What's a good "executive summary" to distinguish what needs to be done here from the low-rent type of named-based virtual hosting.
Taking this question one step further, are NFFC's and sun818's approaches that keep the main domain on a separate IP number somehow related to concerns about enabled name-based virtual hosts, or is there another reason for keeping it separate?
I hope this all makes sense....
[edited by: Robert_Charlton at 6:07 am (utc) on Jan. 7, 2003]
What do they call this kind of account? I'm searching for vocabulary so I can succinctly suggest to this corporate webmaster what he needs to do.
what approach do you take with these multiple domains? Do you promote them separately and after some time you 301 them or they are just there for safeguarding client's domains?
If all domains point to the same content (possibly and probably the same server assuming Apache) use either mod_rewrite or a scripting language to take action according to the setting of the HTTP_HOST environment variable.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?(domain1¦domain2¦domainN)\.tld$
RewriteRule ^(.*) [main-domain.tld...] [R=301,L]
would redirect any requests to www.domain[1-N].tld or domain[1-N].tld to the respective URI on www.main-domain.tld with a 301 server response code.
I´m not sure whether this is what you are after. Let me know if you would like my opinion on the rest of this thread.
Andreas
I´m not sure whether this is what you are after. Let me know if you would like my opinion on the rest of this thread.
Yes, please do comment! Robert_Charlton is trying to find the correct technical terminology to describe "pointing multiple domains to one server using DNS, and 301-redirecting locally on that server if the http_host is not the 'main' domain name - without the use of 'mirrors' or separate hosting accounts for the alternate domain names."
I may be wrong, but I do not believe that this requires name-based or IP-based virtual hosting, since only one 'site' is actually being hosted.
Your command of the precise technical terms is well-known, so the question is, "How can he describe doing this in simple but correct terminology to the site technicians who must implement the changes?"
As you surmise, the goal is to point multiple domains to a single file system, preferably using only one server:
What must be done with the DNS name servers?
What must be done in .httpd.conf?
If access to httpd.conf is denied, can everything be done in .htaccess?
Thanks for the help!
Jim
Yes, please do comment!
I had hoped to escape that with my little piece of code. Let me have dinner first, (set-face-background-pixmap 'default 'marc_10.xpm') and then I will have a look at this thread again.
Andreas
All the additional domains are setup in your dns server and they are all given the IP of the real site.
On the main site, I use mod rewrite to force the server to only deliver the real domain.
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} extradomain1.net$ [OR]
RewriteCond %{HTTP_HOST} extradomain2.com$ [OR]
RewriteCond %{HTTP_HOST} extradomain2.org$
RewriteRule ^(.+) [realdomain.com...] [L,R=301]
It works fine and is very inexpensive. But I will tell you that it is not completely risk free.
Recently, an unamed search engine empolyee was looking through my domain registrations and noticed I owned about 40 similar domains. When these domains were typed in, the all resolved to a single site. (using the method listed above).
That led to the employee telling me that if I didn't stop putting up duplicate sites, he'd have to take action. After explaining that these additional domains where used for print ads and general type-in traffic, and that they weren't actually sites and I didn't ever promote any of the other domains on the web,he seemed o.k. with it, but he did say that in a case like that, they would prefer the domains were on separate sites that had robots.txt exclusions set up.
I think the concern is that in a link analysis system, a 301 from a domain that existed at one time ultimately passes some rank to the new location. That being the case, there is the possibility to be abusive if you intentionally register 100's of expired domains and 301 them all to a single location.
That's not what I was doing and I don't think that is the intent of most dites that use a similar system, but the experience definitely got me thinking more about how stuff may look.
they would prefer the domains were on separate sites that had robots.txt exclusions set up
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?(domain1¦domain2¦domainN)\.tld$
RewriteRule ^/robots.txt$ /robots.txt.other [L]
RewriteCond %{HTTP_HOST} ^(www\.)?(domain1¦domain2¦domainN)\.tld$
RewriteRule ^/(.*) [main-domain.tld...] [R=301,L]
This would send the content of the robots.txt.other when the robots.txt file is requested for any of the other domains.
The red slash is needed in httpd.conf context.
Perhaps a response code of 303 See Other would be more appropriate:
[faqs.org...]
Andreas
First off if you want the domain names to be in any way accessable there need to be proper DNS records for all domain names you want to use. Their A (address) records need to point to the same ip address. Have a look at DNS Oversimplified [rscott.org] for a simple introduction to the Domain Name System.
From what you wrote I believe this to be the case right now. To check lookup the DNS A record using dig domain.tld from the command line.
Once this is the case requests to those domain names will all result in the client connecting to the same server. Since HTTP/1.1 compliant clients are required to send along the host name in the Host field of the HTTP request header the server may treat such a request differently depending on the host name. When you are using name based virtual hosting Apache uses this value to decide which virtual host should handle the request. But this is not necessary if all you want to do is redirect to the main domain. As Jim wrote: only one 'site' is actually being hosted.
All that would need to be done to achieve the redirect is to add the code I posted in msg#18 [webmasterworld.com] to the httpd.conf file. If that is not possible simply put it into the .haccess file in the domain´s root directory.
Andreas
Very cool fix. That is exactly what I need to do. Serve different robots files based on the domain. That should make everyone happy.
I'll try it out and let you know how it works.
I'm assuming the question of HTTP/1.1 compliance isn't that much of an issue at this point.
Is there any consensus about a 303 versus a 301?
Note well the part about pre-HTTP/1.1 user-agents not understanding a 303.
Jim
Note well the part about pre-HTTP/1.1 user-agents not understanding a 303.
That argument does not figure into the equation, since any host name based solution (many domains map to a single ip address) will fail for pre-HTTP/1.1 clients anyway. (See my note in msg#19 [webmasterworld.com])
When suggesting a 303 response instead of the usual 301 code my reasoning was as follows:
A certain resource has been available for a certain time under one URI. This is changed then later on. To let user-agents know about this change in address they send the 301 code. When the change is only temporary they will answer with a 302 code. In the situation at hand it is not so much that a certain resource has moved. What you really want to tell user-agents is that the answer to their request should be requested from this other URI.
In a sense returning a 303 response is less specific than either 301 or 302. You do not give a reason why there is this other URI that the user-agent should use. All you are saying is use it.
In that sense a 303 response addresses the concerns of the SE rep mentioned in WebGuerrilla´s post as well. Spiders should not attribute any PR or links or any other property of the "old" (i.e. the URI that got requested first, the other URIs) to the "new" (i.e. the main URI/domain) URI.
Note: I have no experience how spiders actually treat 303 response codes. My advise is solely based on the language of the HTTP spec.
Andreas
a) the HTTP/1.1 question
Andreas - I understand why, with your approach, you're saying that we might as well use 303s anyway. The question of Netscape 4.x would bother me, though, if the main domain wouldn't work on these browsers.
But, I'm wondering, would the main domain work on pre-HTTP/1.1 clients? If so, I could probably live with the redirected domains not working, since this would affect so few users that the number of errors would be acceptable.
Also, is there a way to handle these errors so a 404 page for the main domain is invoked, or does that require HTTP/1.1 compliance?
b) is name-based virtual hosting involved?
The question of whether name-based virtual hosting is necessary here has been raised several times but not explicitly answered. Is it used in the solutions Andreas and WG are suggesting? This is the nagging question that represents the block I need to get past with the client's web guy. We've talked around it, but I don't see a definitive answer on the thread.
c) the two IP solution
Since the company I'm dealing with doesn't want to go near named-based virtual hosting, at least not in the traditional sense, if it is required in Andreas' and WG's solution, I may well not be able to talk them into it for the main domain. I may be able to talk them into it for redirects... then redirecting that domain, as NFFC suggests, to the main domain.
Is avoiding name-based virtual hosting perhaps why NFFC, in msg#5, is suggesting two IPs... one for the redirects, and one to give the main domain its own IP?
All domains will work work for pre-HTTP/1.1 clients. When looking up the IP address of the server to connect those clients will get the ip address like any other clients as well. Then they will go on and connect to the server that is identified by the IP they got from the nameserver and request the resource they want to get.
Lets have a look what happens when a UA requests an URI like this [aaron-carter.com...] one.
......... pre-HTTP/1.1 ..... ¦ ..... HTTP/1.1-compliant .
.
- look up aaron-carter.com . ¦ - look up aaron-carter.com
.
- connect to 213.210.18.9 .. ¦ - connect to 213.210.18.9
.
- GET / HTTP/1.0 ........... ¦ - GET / HTTP/1.1
. .......................... ¦ . Host: aaron-carter.com
. User-agent: Opera/0.16 ... ¦ . User-agent: Opera/16
. \n ....................... ¦ . \n
.........................................................
As you can see from this example the big difference is that pre-HTTP/1.1 clients do not tell the server which host they connected to. In those days each domain had its own IP and people thought it strange that they should tell the server who he is since they just looked up the IP address and the server should know who he is. With multiple domains hosted on one IP address it is neccessary to tell the server which host you want.
Think of it this way. In the old days there were only single family households. You dialed that household´s telephone number and got the person living there. Today there are large families living together in one home with just a single telephone number. This number does not sufficiently identify the person you actually want to talk to. You need to tell the person that answers the phone who you want to talk to. In the future with IPv6 everybody in those large families will get her own number again and the host field hack of the HTTP/1.1 spec will become obsolete.
When you don´t know who you want to talk to, you can always talk to the person that first answered the phone. That is what the pre-HTTP/1.1 client will do.
So unless your client is already on a server that uses name based virtual hosting and his server is not the one first answering the phone, i.e. not the main server or the default/catch-all virtual server all domains will work even for pre-HTTP/1.1 clients. To them your domains will look like mirrors. That should not matter to you, since all spiders need to support HTTP/1.1 if they want to be successful in today´s server world.
Also, is there a way to handle these errors so a 404 page for the main domain is invoked, or does that require HTTP/1.1 compliance?
Since pre-HTTP/1.1 clients will see the domains as mirrors there should be no errors.
To be precise name-based virtual (redirecting) is involved. As pointed out by Jim (and furthered in my post #19 [webmasterworld.com]: But this is not necessary if all you want to do is redirect to the main domain. As Jim wrote: only one 'site' is actually being hosted.), the redirected domains are not hosted in the sense that there is a webserver that should serve requests made to these domains.
Only if you think in pre-HTTP/1.1 terms could you think of the situation at hand as name-based (none-virtual) hosting. But that is only because those clients simply do not tell the server which host they want.
If that client´s web guy continues to insist on knowing whether this is name-based virtual hosting tell him that, as Ronald Dworking pointed out (numerous others did as well, but for some reason it is alsways Dworking that gets cited here, probably because of people´s {mine included} lack of knowledge in classical greek philosophy) "it is better to look at theories [or concepts] than labels".
It does not really matter whether name-based virtual hosting is involved. All that really matters is whether the employed solution gets done what it should. In lack of a positive assertation it is helpful to look at what might go wrong when using the solution to be evaluated.
Brett wrote in Successful Site in 12 Months with Google Alone I) Put it Online. [webmasterworld.com]
Don't go with virtual hosting - go with a stand alone ip.
He went on to say that he did not trust SE spiders to get it right. However that might be, everybody is well entitled to their mistrusting spiders ability to talk HTTP/1.1 but if you do, you need to be prepared to pay the price of single IP per domain hosting.
It is true, when a spider messed up in the usual shared hosting setting the situation could be devestating to the businesses hosting their sites there. Imagine the serious, respectable law firm specializing in Constitutional Law appearing in SERPS when people search for Aaron Carter and the teen magazine´s site in SERPS for Brandenburg v. Ohio, 395 US 444 (1969) or Barbara Curley and Robert Curley, Administrators of the Estate of Jeffrey Curley v. NAMBLA, et. al.
But if only your client´s domains resolve to the same IP then the worst that could happen would be that their site would be found under a different domain name. This would of course not affect people who went directly to their domain. While this may happen I believe it to be highly improbable. SEs just cannot afford to mess up such an important task. Of course it could still happen.
Again the family example might help. Usually the name-based virtual hosting is like a lot of strangers living together in an apartment house with no locks on the apartment doors. There are only names and when you believe you have the right name you just enter. It is dark and you had a long day. You just go to the bedroom undress and go to bed. The next morning you awake next to AC instead of BS. Now depending on your preferences this might be quite embarassing.
Now your clients situation is more like the guy who has two boyfriends/girlfriends living with him in the same house. One of those he likes better and usually spends more nights with. Now since he errs he goes to the wrong person and awakes the next morning. While he would have enjoyed spending the night with the other person more, this one was ok too.
...does not involve name-based hosting indeed. But that is only due to the fact that all you need to distinguish between are two cases: a) requests to the main domain, b) requests to other domains. And two IP addresses are sufficient for that task. However, as soon as you look at what other domains were being used, e.g. to measure the success of certain campaigns which use some of the other domains, you are in the grey area I described as name-based virtual (redirecting).
Perhaps the two IP solution would be a nice compromise between security and hosting costs. If there is a mess-up then that will most likely only affect the other domains and for the time being you would be unable to measure exactly which other domains are used.
I hope this helps.
Andreas
It certainly does. Thank you! :)
...when a spider messed up in the usual shared hosting setting the situation could be devestating to the businesses hosting their sites there. Imagine the serious, respectable law firm specializing in Constitutional Law appearing in SERPS when people search for Aaron Carter and the teen magazine´s site in SERPS for Brandenburg v. Ohio, 395 US 444 (1969) or Barbara Curley and Robert Curley, Administrators of the Estate of Jeffrey Curley v. NAMBLA, et. al.
Good one, Andreas! :)
Jim
