homepage Welcome to WebmasterWorld Guest from 54.196.198.213
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robots.txt for Wordpress and Vbulletin together - Help
Robots.txt for Wordpress and Vbulletin - Help
anticipation




msg:4356585
 7:31 am on Aug 30, 2011 (gmt 0)

Is it a good idea to have separate robots.txt for wordpress and vbulletin installation on the same domain ?

I have a site where i use wordpress for blogs and a forum by vbulletin. I chose this strategy as vbulletin cms And wordpress forum are bad at seo.

I'm currently facing two problems.

1. Mysite.com is a wp site installed in root /
2. Created a subdomain forum.mysite.com for forum and (vb) installed in /forum directory

But whenever i visit forum.mysite.com the url is same. On clicking internal links, it turns out to be mysite.com/forum/somelink

Because of this setup i can have only one robots.txt

How do I fix this ? What i want is like that of XDA Developers.

XDA has [forum.xda-developers.com...] for vbulletin

and [xda-developers.com...] for Wordpress

It would be great if anyone can guide me in achieving this.

 

phranque




msg:4357029
 8:58 am on Aug 31, 2011 (gmt 0)

welcome to WebmasterWorld, anticipation!

you can only have a single robots.txt file per hostname and it must be in that site's root document directory.

But whenever i visit forum.mysite.com the url is same. On clicking internal links, it turns out to be mysite.com/forum/somelink

this is the problem you must resolve.
all your internal links must refer to the forum subdomain and any external requests for the forum subdirectory must be 301-redirected to the forum subdomain.

except then i found this gem in the vbulletin documentation...
https://www.vbulletin.com/docs/html/technical_vb4_installinsubdirectory [vbulletin.com]:
We do not currently support subdomains due to the way vBulletin handles AJAX calls and cookies.

anticipation




msg:4357110
 1:17 pm on Aug 31, 2011 (gmt 0)

Thanks for the help. Not sure if you saw the two different robots.txt of XDA developers. I'm not sure how they are handling it. Anyway would like to know if with the one Robots.txt , am i at a disadvantage since the crawlers have to crawl lot many pages and there are ONLY couple of links on my wordpress page linking my forum.

g1smd




msg:4357117
 1:24 pm on Aug 31, 2011 (gmt 0)

We do not currently support subdomains due to the way vBulletin handles AJAX calls and cookies.
I think you might have something out of context or else the VB people have used the wrong terminology. www.example.com is a subdomain. What are they trying to say?
phranque




msg:4357387
 10:37 pm on Aug 31, 2011 (gmt 0)

that's why i called it a gem - it is an unqualified statement that couldn't possibly be more confusing or unhelpful.

however, i do believe the OP's question is answered on that page:
Settings -> Options -> Site Name / URL / Contact Details.

anticipation




msg:4358326
 5:10 pm on Sep 3, 2011 (gmt 0)

I have another question on handling sitemaps. Now I have two sitemaps generated by Wordpress and by vbulletin. Is it ok to have two sitemaps?

Or is there anyway I can combine them or something? Please suggest.

phranque




msg:4358429
 11:17 pm on Sep 3, 2011 (gmt 0)

example of multiple sitemaps here:
http://www.sitemaps.org/protocol.php#informing

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved