Welcome to WebmasterWorld Guest from 54.205.75.60

Forum Moderators: bill & werty

Message Too Old, No Replies

Duff feed URL with unwanted amp; part within.

Googlebot redirected.

   
10:18 pm on Jun 27, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The vast majority of subscribers have subscribed via this URL:

www.example.com/pagename?feed=atom&param=value



When this User Agent visits the site, it reports the number of subscribers and the feed ID as part of the UA string:

"Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; nnn subscribers; feed-id=nnnnnnnnnnnnnnnnnnn)"



However the same UA also requests this other malformed URL:

www.example.com/pagename?feed=atom&param=value


from time to time, and it has a very small number of subscribers. This URL has amp; in it, not just the & part.


A few weeks ago a 301 redirect was installed to remove the "amp;" part of the requested URL. Google appears to follow the redirect requesting both URLs in turn; the one with "amp;" followed immediately by the one without it, both using the feed ID with the small number of subscribers.


The point is, the "faulty" URL continues to be requested, and now there are two separate feed IDs that request the exact same URL, one natively and the other redirected.


How long does it take for the redirected URL to no longer be requested? Does it require the subscribers to alter something in their feed reader, perhaps unsubscribe and resubscribe, or will things eventually update themselves automatically?
6:41 am on Sep 27, 2011 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Are you still experiencing this?
7:14 am on Sep 27, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yes. I tried redirecting for several months. The requests for the wrong URL version continued several times per day.

For the last month or two I have been rewriting the request to a static XML file with one message in it "Please unsubscribe from this feed and resubscribe using the URL version without amp; in it". The requests continue and the subscriber number remains static.

I also tried something else for about one month in between those two, but can't remember what it was.
7:36 am on Sep 27, 2011 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Obviously the fetcher doesn't recognize the redirect then. I guess you've tried 404 or 410 errors as well?
8:32 am on Sep 27, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



That might be what I tried in the month that I can't remember what I did. The requests kept on coming whatever it was I did.

There's a good reason for not using 404 or 410. I don't want to lose the subscribers. I have assumed that if they see those types of error, they will assume the site has closed down.

Sending a "please resubscribe using the correct URL" message hasn't worked, so maybe I'll revert to using the 301 redirect instead.
1:45 am on Sep 28, 2011 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



From [google.com...]
Feedfetcher does not follow robots.txt guidelines.

If your feed is publicly available, Google can't restrict users from accessing it. One solution is to configure your site to serve a 404, 410, or other error status message to user-agent Feedfetcher-Google.


It's understandable that you don't want to lose the subscribers. Maybe adding a large eye-catching image to the Please unsubscribe message might draw more attention.