Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Two website urls in single robots.txt

         

raj1094

9:11 am on May 27, 2010 (gmt 0)

10+ Year Member



Dear Webmasters,

Consider if I have two sites, site1 and site2.

site1: Receives very good traffic and if any pages added to the site get index very fast by Google. Google bots crawl the site very often.

site2: Low performing site.

My Question is?

Can we follow this

IN site1 robots.txt can we place as

User-agent: *

Sitemap: http://www.site1/sitemap.xml

sitemap: http://www.site2/sitemap.xml

The above two urls are in same site1 robots.txt, can we follow this, will this be any black hat SEO

[edited by: tedster at 9:19 am (utc) on May 27, 2010]
[edit reason] make example sitemap urls visible [/edit]

AnkitMaheshwari

10:01 am on May 27, 2010 (gmt 0)

10+ Year Member



IMO, a robots.txt file is strictly for the site where it resides, so adding other website in it should not make any difference at all.

Probably you can try adding it and see what your Google webmaster tool has to report about it.

jdMorgan

11:57 am on May 27, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



> The above two urls are in same site1 robots.txt, can we follow this, will this be any black hat SEO?

It will likely be considered to be an error -- a sign of low "technical quality" -- rather than "black hat SEO."

If these two sites are on the same theme, they are competing with each other for links, visitors, and rankings. So if that is the case, serious thought should be given to merging their content, and 301-redirecting the low-performing domain's pages to the higher-performing domain's pages. There is little or no benefit in trying to compete with yourself... especially if you already have other 'real' competition.

Jim