Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

GWT Not Found Errors - Adding Feed at End of Post Name

         

alika

1:07 pm on Sep 2, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have received a notification in GWT about an increase in Not Found errors.

The Not Found errors are of this format:

www.URL.com/year/month/post-namefeed


where the actual URL is of this format

www.URL.com/year/month/post-name
(without "feed" at the end of the post name)

My site has the following:

- Robots.txt file has the command: Disallow: */feed/
- GWT's URL Parameter has been configured that for the parameter "feed" No URLs should be crawled

But I think my above settings are for the URLs www.URL.com/year/month/post-name/feed. But the error GWT is reporting has the word "feed" immediately added to the post name

How should I fix this and prevent Google from adding "feed" at the end of post name?

Thanks

aakk9999

1:30 pm on Sep 2, 2013 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It seems that the URL with feed was created by some kind of error as the word 'feed' is just appended to the URL. This format of URL would indicate to me a problem with internal linking creation where relative path is used.

When you look at the 'linked from' information in WMT, does it say where such links are coming from? If they are coming from your site, you may want to check the linked from page to see if you can spot the error.

The Disallow directive you have in robots.txt will not block these URLs as the directive specifically blocks URLs with the pattern /feed/ and these URLs that return 404 do not have slash before and after the word feed.

To confirm this, enter the problem URL into WMT in section where you can test your robots.txt and you will see that such URLs that return 404 are allowed.

WMT parameter section setup is here irrelevant as 'feed' is not a parameter in your URL sample.

I would let your server return 404 for these and fix an error that causes these URLs to be created in the first place.