homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / RSS, ATOM, and Related Technologies
Forum Library, Charter, Moderators: bill & werty

RSS, ATOM, and Related Technologies Forum

Duff feed URL with unwanted amp; part within.
Googlebot redirected.

 10:18 pm on Jun 27, 2011 (gmt 0)

The vast majority of subscribers have subscribed via this URL:


When this User Agent visits the site, it reports the number of subscribers and the feed ID as part of the UA string:

"Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; nnn subscribers; feed-id=nnnnnnnnnnnnnnnnnnn)"

However the same UA also requests this other malformed URL:


from time to time, and it has a very small number of subscribers. This URL has amp; in it, not just the & part.

A few weeks ago a 301 redirect was installed to remove the "amp;" part of the requested URL. Google appears to follow the redirect requesting both URLs in turn; the one with "amp;" followed immediately by the one without it, both using the feed ID with the small number of subscribers.

The point is, the "faulty" URL continues to be requested, and now there are two separate feed IDs that request the exact same URL, one natively and the other redirected.

How long does it take for the redirected URL to no longer be requested? Does it require the subscribers to alter something in their feed reader, perhaps unsubscribe and resubscribe, or will things eventually update themselves automatically?



 6:41 am on Sep 27, 2011 (gmt 0)

Are you still experiencing this?


 7:14 am on Sep 27, 2011 (gmt 0)

Yes. I tried redirecting for several months. The requests for the wrong URL version continued several times per day.

For the last month or two I have been rewriting the request to a static XML file with one message in it "Please unsubscribe from this feed and resubscribe using the URL version without amp; in it". The requests continue and the subscriber number remains static.

I also tried something else for about one month in between those two, but can't remember what it was.


 7:36 am on Sep 27, 2011 (gmt 0)

Obviously the fetcher doesn't recognize the redirect then. I guess you've tried 404 or 410 errors as well?


 8:32 am on Sep 27, 2011 (gmt 0)

That might be what I tried in the month that I can't remember what I did. The requests kept on coming whatever it was I did.

There's a good reason for not using 404 or 410. I don't want to lose the subscribers. I have assumed that if they see those types of error, they will assume the site has closed down.

Sending a "please resubscribe using the correct URL" message hasn't worked, so maybe I'll revert to using the 301 redirect instead.


 1:45 am on Sep 28, 2011 (gmt 0)

From [google.com...]
Feedfetcher does not follow robots.txt guidelines.

If your feed is publicly available, Google can't restrict users from accessing it. One solution is to configure your site to serve a 404, 410, or other error status message to user-agent Feedfetcher-Google.

It's understandable that you don't want to lose the subscribers. Maybe adding a large eye-catching image to the Please unsubscribe message might draw more attention.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Code, Content, and Presentation / RSS, ATOM, and Related Technologies
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved