Videos are divided into categories so the plan is to have a sitemap index pointing to the actual sitemaps for each category.
sitemap index will reside on the root but our developers want to host the actual sitemap files on the CDN (under media.domain.com).
Is this possible and what will the robots.txt need to include so the end result will be: 1. sitemap index will be on domain.com/sitemap_index.xml 2. sitemap index will point to the individual category sitemaps on media.domain.com/sitemap_cat1.xml, media.domain.com/sitemap_cat2.xml etc. 3. media.domain.com/sitemap_cat1.xml, media.domain.com/sitemap_cat2.xml will list videos on domain.com/video1.php, domain.com/video2.php etc.
Msg#: 4367784 posted 11:16 am on Sep 28, 2011 (gmt 0)
the root robots.txt refers to a sitemap (index?) on the root hostname which only refers to urls on the root hostname.
the subdomain robots.txt refers to its sitemap index on the root hostname. the CDN subdomain's sitemap index only refers to the individual category sitemaps on the CDN subdomain's hostname. so far, so good. the part where i think you are going to have an issue (see #3 above) is that these individual category sitemaps may only refer to urls on the CDN subdomain's hostname.
this is all exactly what is described at the url i provided above.
Msg#: 4367784 posted 8:29 am on Oct 17, 2011 (gmt 0)
How will 301 redirects come in play here?
meaning everything will be "normal": example.com/sitemap_index.xml will be an index file pointing to: example.com/sitemap_cat1.xml, example.com/sitemap_cat2.xml etc...
but example.com/sitemap_index.xml will have a 301 to media.example.com/folder1/folder2/sitemap_index.xml and example.com/sitemap_cat1.xml will have a 301 to media.example.com/folder1/folder2/itemap_cat1.xml etc..
And example.com/robots.txt will have: sitemap: example.com/sitemap_index.xml
Msg#: 4367784 posted 9:50 am on Oct 19, 2011 (gmt 0)
why would you redirect requests for your sitemaps?
request from the developers.
in any case, if anyone is facing similar problem - issue was resolved by having the index file on the root (example.com/sitemap_index.xml) which pointed to the sitemap files on the root but they redirect to the CDN. WMT seems happy with it