Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: goodroi
I'm not convinced that Google isn't punishing us for duplicate content, so I'd like to block the bots from this subdomain so as far as they're concerned, it doesn't exist.
We put a robots.txt file in www.example.com/mobile with the following:
It seems to have no effect. What should we do to disallow this subdomain?
2) Do you have a proper 301 redirect from the URLwww.example.com/mobile/<anything> to mobile.example.com/<anything> to prevent duplicate content problems?
[edited by: jdMorgan at 3:46 am (utc) on Dec. 15, 2008]
1) We do have user agent detection, so if the client is on a mobile device, they should see the mobile version of our site.
2) Good question, and something it looks like we need to get in order. www.example.com/mobile/<anything> is not properly redirecting to mobile.example.com/<anything>. In fact, we don't have an .htaccess file in the mobile folder, which we probably should. This could be making a huge mess, as Google could be seeing the same content on 3 different pages:
Could this be causing the bots to ignore robots.txt as well?