Welcome to WebmasterWorld Guest from 18.104.22.168 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe and Support WebmasterWorld
Robots.txt for Wordpress and Vbulletin together - Help Robots.txt for Wordpress and Vbulletin - Help anticipation msg:4356585 7:31 am on Aug 30, 2011 (gmt 0) Is it a good idea to have separate robots.txt for wordpress and vbulletin installation on the same domain ? I have a site where i use wordpress for blogs and a forum by vbulletin. I chose this strategy as vbulletin cms And wordpress forum are bad at seo. I'm currently facing two problems. 1. Mysite.com is a wp site installed in root / 2. Created a subdomain forum.mysite.com for forum and (vb) installed in /forum directory But whenever i visit forum.mysite.com the url is same. On clicking internal links, it turns out to be mysite.com/forum/somelink Because of this setup i can have only one robots.txt How do I fix this ? What i want is like that of XDA Developers. XDA has [ ...] for vbulletin forum.xda-developers.com and [ ...] for Wordpress xda-developers.com It would be great if anyone can guide me in achieving this.
phranque msg:4357029 8:58 am on Aug 31, 2011 (gmt 0)
welcome to WebmasterWorld, anticipation! you can only have a single robots.txt file per hostname and it must be in that site's root document directory.
But whenever i visit forum.mysite.com the url is same. On clicking internal links, it turns out to be mysite.com/forum/somelink this is the problem you must resolve. all your internal links must refer to the forum subdomain and any external requests for the forum subdirectory must be 301-redirected to the forum subdomain. except then i found this gem in the vbulletin documentation... https://www.vbulletin.com/docs/html/technical_vb4_installinsubdirectory [ vbulletin.com]:
We do not currently support subdomains due to the way vBulletin handles AJAX calls and cookies. anticipation msg:4357110 1:17 pm on Aug 31, 2011 (gmt 0)
Thanks for the help. Not sure if you saw the two different robots.txt of XDA developers. I'm not sure how they are handling it. Anyway would like to know if with the one Robots.txt , am i at a disadvantage since the crawlers have to crawl lot many pages and there are ONLY couple of links on my wordpress page linking my forum.
g1smd msg:4357117 1:24 pm on Aug 31, 2011 (gmt 0)
We do not currently support subdomains due to the way vBulletin handles AJAX calls and cookies. I think you might have something out of context or else the VB people have used the wrong terminology. www.example.com is a subdomain. What are they trying to say? phranque msg:4357387 10:37 pm on Aug 31, 2011 (gmt 0)
that's why i called it a gem - it is an unqualified statement that couldn't possibly be more confusing or unhelpful. however, i do believe the OP's question is answered on that page:
Settings -> Options -> Site Name / URL / Contact Details.
anticipation msg:4358326 5:10 pm on Sep 3, 2011 (gmt 0)
I have another question on handling sitemaps. Now I have two sitemaps generated by Wordpress and by vbulletin. Is it ok to have two sitemaps? Or is there anyway I can combine them or something? Please suggest. phranque msg:4358429 11:17 pm on Sep 3, 2011 (gmt 0)
example of multiple sitemaps here: http ://www.sitemaps.org/protocol.php#informing