Msg#: 3823687 posted 1:22 pm on Jan 8, 2009 (gmt 0)
i have a confusion about sitemap entry in robots.txt file. suppose i have two domains - domain1 and domain2. now let's suppose that domain1 is 9 yrs old while domain2 is just 2 weeks old. now, if i write the sitemap entry of domain2 in the robots.txt file of domain1, will it have any effect.If yes, what and how both the domains will be effected from SEO perspective.
Msg#: 3823687 posted 3:00 pm on Jan 12, 2009 (gmt 0)
I am not sure if I understood your question. I think you are asking if you refer to another domain in a robots.txt file will it help improve rankings. In my experience there will be no significant ranking improvement and you are probably increasing the chance of breaking your robots.txt file.
Most robots.txt files have no links pointing to them and thus they have no significant link popularity to pass. The only possible benefit I could think of is that there is a chance that the search engines may increase the indexing of the domains. I do not see how referring to another domain will help pass along a significant ranking boost or make the search engines think the site is an old, trusted authority site. In my opinion it would be much more beneficial to increase the quantity and quality of your inbound links for both domains.
If you were asking a different question please let me know.
Msg#: 3823687 posted 12:56 pm on Jan 13, 2009 (gmt 0)
you got my question absolutely correct. can you also please tell me if it is okay to enter a new website's name in already established website robot's file to make indexing faster. i hope it will not have any adverse affect.
Msg#: 3823687 posted 2:24 pm on Jan 13, 2009 (gmt 0)
What you are asking to do is not what robots.txt is intended for and could possibly cause issues with some bots. To be safe make sure to validate your robots.txt file and to monitor it.
The best way to increase indexing is to increase the quantity and quality of websites linking to yours. If a site is 100% indexed it may still not deliver any traffic. In general traffic goes up when ranking goes up and ranking goes up when link popularity goes up.