site.com and 2nd.site.com and SEO

[BobMane]

this is for an ecomm site

so if i have site.com

and want to make a live clone 2nd.site.com

will penalty or something bad as a result?

note: I will be taking orders on both sites and trying different backend stuff on each. both will have same branding. overtime i will get rid of 2nd.site.com

[backdraft7]

clone? duplicate content comes to mind...

[BobMane]

wow i guess i did not consider dup content...

it would have some cosmetic differences but most would be the same data.

but i guess the question then was about the url. dont i have the right to have same data if using 2nd.site.com which is in the end actually site.com?

[HuskyPup]

You have the right to do whatever you want to do, just don't expect it to rank because you have a different design with precisely the same content.

[BobMane]

yes well i have to do this for backoffice tech reasons but want to do it without bad impact.


note: this is also in magento. its a bit hard to understand how many company might have multistore selling same thing at diff price point and dont get flagged?


so does anyone know what or how much need to be changed to avoid dup content flag?

can i just change the metadata (description, etc) or all content

[netmeg]

Why would Google want to crawl and index multiple copies of the exact same content?

I work extensively with Magento; I know how the multi store stuff works. When we have multiple stores with the same items, we only ever let one into Google. Seriously. Anything else and you risk killing them both.

[TheOptimizationIdiot]

yes well i have to do this for backoffice tech reasons but want to do it without bad impact.

Why not just password protect it then?

[BobMane]

"When we have multiple stores with the same items, we only ever let one into Google. Seriously. Anything else and you risk killing them both. "

1: are you talking about multi with site.com and site2.com or 2nd.site.com

2: are you saying this on site2.com or 2nd.site.com
: Navigate to Configuration -> Design -> HTML Head and set the Default Robots option to �NOINDEX, NOFOLLOW�. This should add a �NOINDEX, NOFOLLOW� meta tag to your site.....

....is all i need to do to not be penalized?

3: any difference if its site2.com or 2nd.site.com?

[edited by: BobMane at 5:16 am (utc) on Apr 22, 2013]

[BobMane]

"Why not just password protect it then? "

no i am selling stuff on both, they will be public. my main site and the other site will be similar...not exact clone.

[TheOptimizationIdiot]

Well, the only way to do it so it's accessible is like was suggested with noindex,nofollow (I'd personally serve it in the server header subdomain-wide) and I'd add rel="canonical" pointing to the corresponding page on the main domain.

Hopefully, you understand the about the only way you will get traffic to the subdomain variation of the site is through a direct link click, because the search engines will not index it with what's being suggested and so many people "just type the name they see" into a search engine unless they click a link (or don't have a clue what a subdomain is and just type example.com if they type a domain in the browser), so if you're trying to track how different things work I don't see being an effective enough way to take any risk with a site that's ranking personally, because it doesn't sound like you're going to get significant traffic to it and I would definitely recommend finding another way to split test different things (like either a JavaScript or cookie triggered switch), even if it may not be as easy.

Also, similar / not exact clone doesn't really matter. You can reason with us all you want and say it's for some internal purpose and not exactly the same, but the algo's totally unreasonable and a near duplicate you launch and try to get indexed could range from totally ineffective and "ignored" by search engines (at best) or it could really hose your main domain if the algo decides it's to add extra pages and more chances at visitors (spam).

There's no "duplicate content" penalty specifically, but there are plenty of "looks like spam" based penalties and possible negative ranking impact scenarios from near duplicates and duplicates (which I believe increase when it's not www v non-www exact duplication) I wouldn't use a second near duplicate subdomain for anything personally.

[Robert Charlton]

What TOI said. If this is for testing backend stuff, some sort of split testing may be the way to go. Under no circumstances would I put the same content on two domains if I cared about performance in search.

[BobMane]

in this case it sounds like i will just use totally different products on 2nd site to avoid issues. so if i duplicate my magento site (just for the theme and plugins + admin config)....then I add new products - while have no old products. its not considered same site or it still is?

what else do i need to change?

[netmeg]

Well you have to think in terms of percentages (because Google probably does) How much of the overall content (including navigation, images, text, products, footers and headers - EVERYTHING) is duplicated from one site to the other? If a majority of the overall content is unique, you're probably okay. If only a small part of the content (a few products) is unique, then you're probably not.

I have six WordPress sites (one each in six states) that have essentially the same templates, styles, type of navigation, plugins and policy pages, but each site has 500+ unique events, and that means that probably 80% of each of my sites are unique, and maybe 20% is the same. And so far (fingers crossed) that has worked for me.

But there's no set percentage that I can tell you that guaranteed works. Search engines don't work like that. I can only tell you my situation.

Start with a small amount of duplicated content (very small) and test it till you find your threshold of pain.

[BobMane]

how do you if you are penalized

[BobMane]

opps...meant to say...how do you know if you are penalized ?

[netmeg]

Traffic from Google stops.

Note, you won't be penalized for duplicate content; your sites may still be in Google. But you'll be filtered, which has the same effect - basically they'll think one or both of your sites is too low quality to rank.

[newsnshop]

I have seen a site using same database for both main site and sub domain and ranking well. Sometime search results show both for same query.

[BobMane]

still never figured this out...what - exactly - makes a page different?

meta
keywords
images
text
everything
only somethings
/?

[TheOptimizationIdiot]

what - exactly - makes a page different?

The page, all of it.

If you want to duplicate, run split tests, otherwise, make everything you can unique. Similar to what I said before: You can tell us whatever you want, but the algo is unreasoning, so other than split testing in an obvious way, my best advice is don't duplicate anything.

[tedster]

I can't find the reference right now (it was a video) but I remember that Matt Cutts stated that two URLs need to be "substantially different" not to be tagged as duplicate. Without giving away any secret formula he mentioned the number "around 85%" as a ballpark guess.

If you have just two URLs that are judged to be duplicate, one of them just gets ignored in the SERPs or plave under the "more results" link at the end. Make sense to me. If a site has a substantial number of duplicates, then the entire site might get ignored.

[BobMane]

does it make any difference if i use

site.com and 2nd.site.com
or
site.com and site.com/2
or
site.com and site2.com

...for SEO of site.com(assuming the content of second site is unique enough not to matter)

[tedster]

Bob, I think that decision is more of a long-term user friendliness decision than a technical SEO decision. Long term, user friendliness IS SEO. If you're sure that "overtime i will get rid of 2nd.site.com" then I'd probably lean toward not ever building it. Otherwise, you may be setting up a future bump in the road.

[BobMane]

i have to build it to test 2 different subscription plug ins on my backend. subscription plug ins are quite complex so i need to run them both live and test each.

[TheOptimizationIdiot]

Why can't you leave an empty div for the plug-in display as a "spacer" in the front-end template and then simply populate the div with one plug-in or the other via AJAX and run an a/b test that way?

If not that exactly, the point is, there is very likely a way to a/b test something like that dynamically with AJAX and not need to build another site at all.

[BobMane]

we considered that but for various reasons it to messy but more so to hard to track and update (like how the code interacts with magento etc...) - its kind of a test to see which is better overall code set. becasue one will eventually win, really need to have them separate to keep them as separate self contained code sets.

[TheOptimizationIdiot]

code-set1.example.com
-- .htaccess
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^some-custom-string-you-set$
RewriteRule .? http://www.example.com%{REQUEST_URI} [R=301,L]

# Whatever else you normally run to make code-set1 function

code-set2.example.com
-- .htaccess
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^some-custom-string-you-set$
RewriteRule .? http://www.example.com%{REQUEST_URI} [R=301,L]

# Whatever else you normally run to make code-set2 function

www.example.com
-- .htaccess
RewriteEngine on
RewriteRule ^a-b-test\.php - [L]
RewriteRule \.(css|js|jpe?g|gif|other-stuff-you-dont-canonicalize) - [L]

RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule .? http://www.example.com%{REQUEST_URI} [R=301,L]

# Whatever your URL extension is for page requests
RewriteRule \.ext$ /a-b-test.php [L]

-- /a-b-test.php

<?php
error_reporting(0);
ini_set('user_agent','some-custom-string-you-set');
# I'd run a reverse/forward look-up for search bot UA's here and default to code-set1

$show=mt_rand(1,2);
if($show===2 || $_COOKIE['to_show']==2) {
# Set Cookie to_show=2 here
echo file_get_contents('http://code-set2.example.com'.$_SERVER['REQUEST_URI']);
}
else {
echo file_get_contents('http://code-set1.example.com'.$_SERVER['REQUEST_URI']);
}

# If you want to track how many times each is shown

ob_flush();

# Connect to db here (obviously) lol

$track='UPDATE table SET code_set_shown_'.$show.'=code_set_shown_'.$show.'+1';
mysqli_query($link,$track);

[BobMane]

ok thanks for that info, being only somewhat code savvy...can you tell me what this is doing.

to me, it looks like i am running 2 separate websites in 2 separate folders on server with 2 separate db but .htaccess is:

1: directing users to site1 by default
and
2: telling robots to only look at site1.

[TheOptimizationIdiot]

to me, it looks like i am running 2 separate websites in 2 separate folders on server with 2 separate db but .htaccess is:

Yes, but the .htaccess is actually sending all requests for www.exmaple.com to a php script internally.

---

Basically what I posted is a "PHP Switch" so rather than one code set running on www.example.com and another one running on site2.example.com you move both to their own sub-domain for now.

What you currently have running on www.example.com moves to code-set1.example.com and what you would be putting on site2.example.com is set up on code-set2.example.com.

Then when a visitor requests a page from www.example.com the .htaccess sends that request to the php script which "grabs" the info from either code-set1.example.com or code-set2.example.com randomly and shows the visitor that version of info. (The cookie keeps them visitors from switching from one code set to another to the other in the middle of a visit, so it's random on a per-visit basis rather than on a per-page basis.)

That way only www.example.com is accessible externally (to visitors or search engines), but internally you're switching back and forth between what code set is shown to users who visit www.example.com for testing purposes.

So, in short:
Externally, all visitors are sent to www.example.com

Internally, you're accessing either code-set1.example.com or code-set2.example.com via PHP and showing one or the other to the visitor on www.example.com for split testing.

Hope that makes sense.

[BobMane]

get it, sounds like a good approach. however, we need users to be able to go to either site. e.g. user1 subscribed on site1 last year but can now subscribe on site 2 also. user1 can go to either site anytime. even though we are doing testing to make it sort of have some other logical public reason, we will offer limited/discount products on site2

[TheOptimizationIdiot]

1.) Update this line --> if($show===2 || $_COOKIE['to_show']==2) {

to: if(($show===2 && $_COOKIE['to_show']!=1) || $_COOKIE['to_show']==2) {

2.) When someone logs in on site1 (or site2), give them a link to the other site and run it through a redirect that sets the cookie to_show=siteN. (So, user signs up on site1 or is a current user == They login and are shown site1 by default. User signs up on site2 and logs in == They are shown site2 by default.)

On each version of the site, put a link to the other that runs through a redirect.

So, the .htaccess changes a bit to:

www.example.com
-- .htaccess
RewriteEngine on
RewriteRule ^a-b-test\.php - [L]
RewriteRule ^show-site-(1|2)\.php - [L]
RewriteRule \.(css|js|jpe?g|gif|other-stuff-you-dont-canonicalize) - [L]

RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule .? http://www.example.com%{REQUEST_URI} [R=301,L]

# Whatever your URL extension is for page requests
RewriteRule \.ext$ /a-b-test.php [L]

-- /show-site-2.php

<?php

setcookie("to_show", '2');

header('HTTP/1.1 301 Permanent Redirect');
# Above is to make sure the browser doesn't re-request
# /show-site-2.php on a refresh.

# Link from site1 to /show-site-2.php
# <a href="/show-site-2.php">View Discounts [or whatever]</a>
# change the following location to correspond with the page
# on site2 you want them to land on

header('Location: http://www.example.com/the-page-from-site2-you-want-to-show-them.ext');

?>

-- /show-site-1.php

<?php

setcookie("to_show", '1');

header('HTTP/1.1 301 Permanent Redirect');
# Above is to make sure the browser doesn't re-request
# /show-site-1.php on a refresh.

# Link from site2 to /show-site-1.php
# <a href="/show-site-1.php">View Main Site [or whatever]</a>
# change the following location to correspond with the page
# on site1 you want them to land on

header('Location: http://www.example.com/the-page-from-site1-you-want-to-show-them.ext');

?>

* Something along those lines and we're getting into "technicalities" now that really depend on the exact interaction. There are a ton of different ways to do things, but getting the details right without knowing the exact situation is very difficult, so hopefully you can take the idea and "run with it" a bit.