homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Redireted URls showing up in Robots.txt Errors

 8:55 pm on Apr 1, 2011 (gmt 0)


Their is no single url I am restricting in robots.txt file but still in GWT it showing around 1800 URLS restricted by robots.txt.

The common thing observed in these urls are all redirected to different site (because it is coupons site).

If any threads already exists for this issue please let me know.

Thanks in Advance



 10:58 am on Apr 2, 2011 (gmt 0)

Is your robots.txt file blank?

How are you redirecting to the different sites?


 8:58 am on Apr 13, 2011 (gmt 0)

Does the second site also belong to you? If so, what does its robots.txt say? Are they 301 redirects? If so, I'm pretty sure gwt is giving you the information that applies to the new location, not the original one, so that's where you need to investigate.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved