cmendla - 11:44 pm on Jun 14, 2010 (gmt 0) [edited by: tedster at 1:11 am (utc) on Jun 15, 2010]
I have a google blog on one of my sites. I recently converted from FTP publishing to google publishing. The site is located at blog.example.com.
If I look in my crawl errors I see
URL Detail Detected
http://blog.example.com/search/label/widgets URL restricted by robots.txt Jun 8, 2010
http://blog.example.com/search/label/widgests2 URL restricted by robots.txt May 30, 2010
My robots.txt for the blog subdomain shows:
Does this matter? I'm not sure exactly what it is blocking.. ie just the labels page or a search that results in labels.
Do I need to do anything with this or should I just let things lie peacefully
[edit reason] switch to example.com - it can never be owned [/edit]
[edited by: tedster at 1:11 am (utc) on Jun 15, 2010]